Ultrasonic robotic surgical navigation

ABSTRACT

Surgical robot systems, anatomical structure tracker apparatuses, and US transducer apparatuses are disclosed. A surgical robot system includes a robot, a US transducer, and at least one processor. The robot includes a robot base, a robot arm coupled to the robot base, and an end-effector coupled to the robot arm. The end-effector is configured to guide movement of a surgical instrument. The US transducer is coupled to the end-effector and operative to output US imaging data of anatomical structure proximately located to the end-effector. The least one processor is operative to obtain an image volume for the patient and to track pose of the end-effector relative to anatomical structure captured in the image volume based on the US imaging data.

FIELD

The present disclosure relates to position recognition systems and moreparticularly to end-effector and instrument tracking and manipulationduring robot assisted surgical procedures.

BACKGROUND

Position recognition systems are used to determine the position of andtrack a particular object in 3-dimensions (3D). In robot assistedsurgeries, for example, certain objects, such as surgical instruments,need to be tracked with a high degree of precision as the instrument isbeing positioned and moved by a robot or by a physician.

Infrared signal based position recognition systems may use passiveand/or active sensors or markers for tracking the objects. Objects to betracked may include passive sensors, such as reflective spherical balls,which are positioned at strategic locations on the object to be tracked.Infrared transmitters transmit a signal, and the reflective sphericalballs reflect the signal to aid in determining the position of theobject in 3D. In active sensors or markers, the objects to be trackedinclude active infrared transmitters, such as light emitting diodes(LEDs), and thus generate their own infrared signals for 3D detection.

With either active or passive tracking sensors, the system thengeometrically resolves the 3-dimensional position of the active and/orpassive sensors based on information from or with respect to one or moreof the infrared cameras, digital signals, known locations of the activeor passive sensors, distance, the time it took to receive the responsivesignals, other known variables, or a combination thereof.

Some existing surgical robot systems utilize optical tracking registeredto a medical image as feedback for positioning a robotic arm while alsovisualizing instruments. Surgical procedures using such systems can beperformed relatively quickly and accurately, however the procedureceases whenever blockage occurs in the line of sight from the patientreference tracker to the cameras. Additionally, many surgical workflowswith existing surgical robotic systems require x-rays or computerizedtomography (CT) scans during operation and/or registration procedures.The system and procedure described herein overcomes many of theselimitations.

SUMMARY

Surgical robot systems, anatomical structure tracker apparatuses, andultrasound (US) transducer apparatuses are disclosed.

Some embodiments are directed to a surgical robot system that includes arobot, a US transducer, and at least one processor (“processor”). Therobot includes a robot base, a robot arm coupled to the robot base, andan end-effector coupled to the robot arm. The end-effector is configuredto guide movement of a surgical instrument. The US transducer is coupledto the end-effector and operative to output US imaging data ofanatomical structure proximately located to the end-effector. Theprocessor is operative to obtain an image volume for the patient and totrack pose of the end-effector relative to anatomical structure capturedin the image volume based on the US imaging data.

Some other embodiments are directed to an anatomical structure trackerapparatus includes an optical tracking array and a US transducer. Theoptical tracking array includes a plurality of spaced apart markers. TheUS transducer is rigidly coupled to and spaced apart from the opticaltracking array. The US transducer is output US imaging data ofanatomical structure.

Some other embodiments are directed to a US transducer apparatus thatincludes a wire and a US transducer attached to an end of the wire. Insome further embodiments, the wire comprises a Kirschner wire, and anoptical tracking array having a plurality of spaced apart markers isattached to the rigid wire.

Other surgical robot systems, anatomical structure tracker apparatuses,and US transducer apparatuses according to embodiments of the inventivesubject matter will be or become apparent to one with skill in the artupon review of the following drawings and detailed description. It isintended that all such additional surgical robot systems, anatomicalstructure tracker apparatuses, and US transducer apparatuses be includedwithin this description, be within the scope of the present inventivesubject matter, and be protected by the accompanying claims. Moreover,it is intended that all embodiments disclosed herein can be implementedseparately or combined in any way and/or combination.

DESCRIPTION OF THE DRAWINGS

Aspects of the present disclosure are illustrated by way of example andare not limited by the accompanying drawings. In the drawings:

FIG. 1 is an overhead view of a potential arrangement for locations ofthe robotic system, patient, surgeon, and other medical personnel duringa surgical procedure;

FIG. 2 illustrates the robotic system including positioning of thesurgical robot and the camera relative to the patient according to oneembodiment;

FIG. 3 illustrates a surgical robotic system in accordance with anexample embodiment;

FIG. 4 illustrates a portion of a surgical robot in accordance with anexample embodiment;

FIG. 5 illustrates a block diagram of a surgical robot in accordancewith an example embodiment;

FIG. 6 illustrates a surgical robot in accordance with an exampleembodiment;

FIGS. 7A-7C illustrate an end-effector in accordance with an exampleembodiment;

FIG. 8 illustrates a surgical instrument and the end-effector, beforeand after, inserting the surgical instrument into the guide tube of theend-effector according to one embodiment;

FIGS. 9A-9C illustrate portions of an end-effector and robot arm inaccordance with an example embodiment;

FIG. 10 illustrates a dynamic reference array, an imaging array, andother components in accordance with an example embodiment;

FIG. 11 illustrates operations for registration in accordance with anexample embodiment;

FIG. 12A-12B illustrate embodiments of imaging devices according toexample embodiments;

FIG. 13A illustrates a portion of a robot including the robot arm and anend-effector in accordance with an example embodiment;

FIG. 13B is a close-up view of the end-effector, with a plurality oftracking markers rigidly affixed thereon, shown in FIG. 13A;

FIG. 13C is an instrument or instrument with a plurality of trackingmarkers rigidly affixed thereon according to one embodiment;

FIG. 14A is an alternative version of an end-effector with moveabletracking markers in a first configuration;

FIG. 14B is the end-effector shown in FIG. 14A with the moveabletracking markers in a second configuration;

FIG. 14C shows the template of tracking markers in the firstconfiguration from FIG. 14A;

FIG. 14D shows the template of tracking markers in the secondconfiguration from FIG. 14B;

FIG. 15A shows an alternative version of the end-effector having only asingle tracking marker affixed thereto;

FIG. 15B shows the end-effector of FIG. 15A with an instrument disposedthrough the guide tube;

FIG. 15C shows the end-effector of FIG. 15A with the instrument in twodifferent positions, and the resulting logic to determine if theinstrument is positioned within the guide tube or outside of the guidetube;

FIG. 15D shows the end-effector of FIG. 15A with the instrument in theguide tube at two different frames and its relative distance to thesingle tracking marker on the guide tube;

FIG. 15E shows the end-effector of FIG. 15A relative to a coordinatesystem;

FIG. 16 is a block diagram of operations for navigating and moving theend-effector of the robot to a desired target trajectory;

FIGS. 17A-17B depict an instrument for inserting an expandable implanthaving fixed and moveable tracking markers in contracted and expandedpositions, respectively;

FIGS. 18A-18B depict an instrument for inserting an articulating implanthaving fixed and moveable tracking markers in insertion and angledpositions, respectively;

FIGS. 19A depicts an embodiment of a robot with interchangeable oralternative end-effectors;

FIG. 19B depicts an embodiment of a robot with an instrument styleend-effector coupled thereto;

FIG. 20 depicts a guide tube configured to guide movement of a surgicalinstrument through the guide tube, and an ultrasound (US) transducerunit formed by an array of US transducers spaced apart along a leadingedge of the guide tube, in accordance with some embodiments;

FIGS. 21A-21C depict differently configured surgical instruments whichhave shafts configured to be tracked relative to the guide tube usingthe US transducer;

FIG. 22 depicts the surgical instrument of FIG. 21A at three differentdepths and rotations relative to the guide tube, in accordance with someembodiments;

FIG. 23 depicts a flowchart of operations that can be performed by atleast one processor to track pose of the end-effector relative toanatomical structure captured in an image volume based on US imagingdata from a US transducer, in accordance with some embodiments;

FIG. 24 depicts a flowchart of operations that can be performed by atleast one processor to generate steering information based on a presentpose of an end-effector determined from US imaging data, in accordancewith some embodiments;

FIG. 25 depicts a more detailed flowchart of operations and be performedby at least one processor to generate navigation information that can beused to guide movement of the robot end-effector toward a target pose,in accordance with some embodiments;

FIGS. 26A-C depicts a sequence of snapshots of a robotic arm of thesurgical robot system moving laterally to a target pose whileautomatically maintaining contact between the US transducer and thepatient's skin and normal to the body surface, in accordance with someembodiments;

FIG. 27 depicts a flowchart of operations for controlling movement ofthe robot arm to a target pose using a combination of optical feedbackcontrol and US transducer feedback control, in accordance with someembodiments;

FIG. 28 depicts an anatomical structure tracker apparatus that isconfigured in accordance with some embodiments;

FIG. 29 depicts another anatomical structure tracker apparatus that isconfigured in accordance with some other embodiments; and

FIGS. 30-38 depict various US transducer apparatuses which areconfigured in accordance with some embodiments.

DETAILED DESCRIPTION

It is to be understood that the present disclosure is not limited in itsapplication to the details of construction and the arrangement ofcomponents set forth in the description herein or illustrated in thedrawings. The teachings of the present disclosure may be used andpracticed in other embodiments and practiced or carried out in variousways. Also, it is to be understood that the phraseology and terminologyused herein is for the purpose of description and should not be regardedas limiting. The use of “including,” “comprising,” or “having” andvariations thereof herein is meant to encompass the items listedthereafter and equivalents thereof as well as additional items. Unlessspecified or limited otherwise, the terms “mounted,” “connected,”“supported,” and “coupled” and variations thereof are used broadly andencompass both direct and indirect mountings, connections, supports, andcouplings. Further, “connected” and “coupled” are not restricted tophysical or mechanical connections or couplings.

The following discussion is presented to enable a person skilled in theart to make and use embodiments of the present disclosure. Variousmodifications to the illustrated embodiments will be readily apparent tothose skilled in the art, and the principles herein can be applied toother embodiments and applications without departing from embodiments ofthe present disclosure. Thus, the embodiments are not intended to belimited to embodiments shown, but are to be accorded the widest scopeconsistent with the principles and features disclosed herein. Thefollowing detailed description is to be read with reference to thefigures, in which like elements in different figures have like referencenumerals. The figures, which are not necessarily to scale, depictselected embodiments and are not intended to limit the scope of theembodiments. Skilled artisans will recognize the examples providedherein have many useful alternatives and fall within the scope of theembodiments.

Turning now to the drawing, FIGS. 1 and 2 illustrate a surgical robotsystem 100 in accordance with an example embodiment. Surgical robotsystem 100 may include, for example, a surgical robot 102, one or morerobot arms 104, a base 106, a display 110, an end-effector 112, forexample, including a guide tube 114, and one or more tracking markers118. The surgical robot system 100 may include a patient tracking device116 also including one or more tracking markers 118, which is adapted tobe secured directly to the patient 210 (e.g., to the bone of the patient210). The surgical robot system 100 may also utilize a camera 200, forexample, positioned on a camera stand 202. The camera stand 202 can haveany suitable configuration to move, orient, and support the camera 200in a desired position. The camera 200 may include any suitable camera orcameras, such as one or more infrared cameras (e.g., bifocal orstereophotogrammetric cameras), able to identify, for example, activeand passive tracking markers 118 in a given measurement volume viewablefrom the perspective of the camera 200.

The camera 200 may scan the given measurement volume and detect thelight that comes from the markers 118 in order to identify and determinethe position of the markers 118 in three-dimensions. For example, activemarkers 118 may include infrared-emitting markers that are activated byan electrical signal (e.g., infrared light emitting diodes (LEDs)), andpassive markers 118 may include retro-reflective markers that reflectinfrared light (e.g., they reflect incoming IR radiation into thedirection of the incoming light), for example, emitted by illuminatorson the camera 200 or other suitable device.

FIGS. 1 and 2 illustrate a potential configuration for the placement ofthe surgical robot system 100 in an operating room environment. Forexample, the robot 102 may be positioned near or next to patient 210.Although depicted near the head of the patient 210, it will beappreciated that the robot 102 can be positioned at any suitablelocation near the patient 210 depending on the area of the patient 210undergoing the operation. The camera 200 may be separated from the robotsystem 100 and positioned at the foot of patient 210. This locationallows the camera 200 to have a direct visual line of sight to thesurgical field 208. Again, it is contemplated that the camera 200 may belocated at any suitable position having line of sight to the surgicalfield 208. In the configuration shown, the surgeon 120 may be positionedacross from the robot 102, but is still able to manipulate theend-effector 112 and the display 110. A surgical assistant 126 may bepositioned across from the surgeon 120 again with access to both theend-effector 112 and the display 110. If desired, the locations of thesurgeon 120 and the assistant 126 may be reversed. The traditional areasfor the anesthesiologist 122 and the nurse or scrub tech 124 remainunimpeded by the locations of the robot 102 and camera 200.

With respect to the other components of the robot 102, the display 110can be attached to the surgical robot 102 and in other exampleembodiments, display 110 can be detached from surgical robot 102, eitherwithin a surgical room with the surgical robot 102, or in a remotelocation. End-effector 112 may be coupled to the robot arm 104 andcontrolled by at least one motor. In example embodiments, end-effector112 can comprise a guide tube 114, which is able to receive and orient asurgical instrument 608 (described further herein) used to performsurgery on the patient 210.

As used herein, the term “end-effector” is used interchangeably with theterms “end-effectuator” and “effectuator element.” The term “instrument”is used in a non-limiting manner and can be used interchangeably with“tool” to generally refer to any type of device that can be used duringa surgical procedure in accordance with embodiments disclosed herein.Example instruments include, without limitation, drills, screwdriver s,saws, dilators, retractors, implant inserters, and implants such as ascrews, spacers, interbody fusion devices, plates, rods, etc. Althoughgenerally shown with a guide tube 114, it will be appreciated that theend-effector 112 may be replaced with any suitable instrumentationsuitable for use in surgery. In some embodiments, end-effector 112 cancomprise any known structure for effecting the movement of the surgicalinstrument 608 in a desired manner.

The surgical robot 102 is able to control the translation andorientation of the end-effector 112. The robot 102 is able to moveend-effector 112 along x-, y-, and z-axes, for example. The end-effector112 can be configured for selective rotation about one or more of thex-, y-, and z-axis, and a Z Frame axis (such that one or more of theEuler Angles (e.g., roll, pitch, and/or yaw) associated withend-effector 112 can be selectively controlled). In some exampleembodiments, selective control of the translation and orientation ofend-effector 112 can permit performance of medical procedures withsignificantly improved accuracy compared to conventional robots thatutilize, for example, a six degree of freedom robot arm comprising onlyrotational axes. For example, the surgical robot system 100 may be usedto operate on patient 210, and robot arm 104 can be positioned above thebody of patient 210, with end-effector 112 selectively angled relativeto the z-axis toward the body of patient 210.

In some example embodiments, the pose of the surgical instrument 608 canbe dynamically updated so that surgical robot 102 can be aware of thepose of the surgical instrument 608 at all times during the procedure.Consequently, in some example embodiments, surgical robot 102 can movethe surgical instrument 608 to the desired pose quickly without anyfurther assistance from a physician (unless the physician so desires).

As used herein, the term “pose” refers to the position and/or therotational angle of one object (e.g., dynamic reference array,end-effector, surgical instrument, anatomical structure, etc.) relativeto another object and/or to a defined coordinate system. A pose maytherefore be defined based on only the multidimensional position of oneobject relative to another object and/or relative to a definedcoordinate system, based on only the multidimensional rotational anglesof the object relative to another object and/or to a defined coordinatesystem, or based on a combination of the multidimensional position andthe multidimensional rotational angles. The term “pose” therefore isused to refer to position, rotational angle, or combination thereof.

In some further embodiments, surgical robot 102 can be configured tocorrect the path of the surgical instrument 608 if the surgicalinstrument 608 strays from the selected, preplanned trajectory. In someexample embodiments, surgical robot 102 can be configured to permitstoppage, modification, and/or manual control of the movement ofend-effector 112 and/or the surgical instrument 608. Thus, in use, inexample embodiments, a physician or other user can operate the system100, and has the option to stop, modify, or manually control theautonomous movement of end-effector 112 and/or the surgical instrument608. Further details of surgical robot system 100 including the controland movement of a surgical instrument 608 by surgical robot 102 can befound in U.S. patent application Ser. No. 13/924,505, which isincorporated herein by reference in its entirety.

The robotic surgical system 100 can comprise one or more trackingmarkers 118 configured to track the movement of robot arm 104,end-effector 112, patient 210, and/or the surgical instrument 608 inthree dimensions. In example embodiments, a plurality of trackingmarkers 118 can be mounted (or otherwise secured) thereon to an outersurface of the robot 102, such as, for example and without limitation,on base 106 of robot 102, on robot arm 104, or on the end-effector 112.In example embodiments, at least one tracking marker 118 of theplurality of tracking markers 118 can be mounted or otherwise secured tothe end-effector 112. One or more tracking markers 118 can further bemounted (or otherwise secured) to the patient 210. In exampleembodiments, the plurality of tracking markers 118 can be positioned onthe patient 210 spaced apart from the surgical field 208 to reduce thelikelihood of being obscured by the surgeon, surgical instruments, orother parts of the robot 102. Further, one or more tracking markers 118can be further mounted (or otherwise secured) to the surgicalinstruments 608 (e.g., a screwdriver, dilator, implant inserter, or thelike). Thus, the tracking markers 118 enable each of the marked objects(e.g., the end-effector 112, the patient 210, and the surgicalinstruments 608) to be tracked by the robot 102 via the camera 200. Inexample embodiments, system 100 can use tracking information collectedfrom each of the marked objects to calculate the pose (e.g., orientationand location), for example, of the end-effector 112, the surgicalinstrument 608 (e.g., positioned in the tube 114 of the end-effector112), and the relative position of the patient 210.

The markers 118 may include radiopaque or optical markers. The markers118 may be suitably shaped include spherical, spheroid, cylindrical,cube, cuboid, or the like. In example embodiments, one or more ofmarkers 118 may be optical markers. In some embodiments, the positioningof one or more tracking markers 118 on end-effector 112 can maximize theaccuracy of the positional measurements by serving to check or verifythe position of end-effector 112. Further details of surgical robotsystem 100 including the control, movement and tracking of surgicalrobot 102 and of a surgical instrument 608 can be found in U.S. patentapplication Ser. No. 13/924,505, which is incorporated herein byreference in its entirety.

Example embodiments include one or more markers 118 coupled to thesurgical instrument 608. In example embodiments, these markers 118, forexample, coupled to the patient 210 and surgical instruments 608, aswell as markers 118 coupled to the end-effector 112 of the robot 102 cancomprise conventional infrared light-emitting diodes (LEDs) or anOptotrak® diode capable of being tracked using a commercially availableinfrared optical tracking system such as Optotrak®. Optotrak® is aregistered trademark of Northern Digital Inc., Waterloo, Ontario,Canada. In other embodiments, markers 118 can comprise conventionalreflective spheres capable of being tracked using a commerciallyavailable optical tracking system such as Polaris Spectra. PolarisSpectra is also a registered trademark of Northern Digital, Inc. In anexample embodiment, the markers 118 coupled to the end-effector 112 areactive markers which comprise infrared light-emitting diodes which maybe turned on and off, and the markers 118 coupled to the patient 210 andthe surgical instruments 608 comprise passive reflective spheres.

In example embodiments, light emitted from and/or reflected by markers118 can be detected by camera 200 and can be used to monitor the poseand movement of the marked objects. In alternative embodiments, markers118 can comprise a radio-frequency and/or electromagnetic reflector ortransceiver and the camera 200 can include or be replaced by aradio-frequency and/or electromagnetic transceiver.

Similar to surgical robot system 100, FIG. 3 illustrates a surgicalrobot system 300 and camera stand 302, in a docked configuration,consistent with an example embodiment of the present disclosure.Surgical robot system 300 may comprise a robot 301 including a display304, upper arm 306, lower arm 308, end-effector 310, vertical column312, casters 314, cabinet 316, tablet drawer 318, and ring 324 ofinformation. Camera stand 302 may comprise camera 326. These componentsare described in greater with respect to FIG. 5 . FIG. 3 illustrates thesurgical robot system 300 in a docked configuration where the camerastand 302 is nested with the robot 301, for example, when not in use. Itwill be appreciated by those skilled in the art that the camera 326 androbot 301 may be separated from one another and positioned at anyappropriate pose during the surgical procedure, for example, as shown inFIGS. 1 and 2 .

FIG. 4 illustrates a base 400 consistent with an example embodiment ofthe present disclosure. Base 400 may be a portion of surgical robotsystem 300 and comprise cabinet 316. Cabinet 316 may house certaincomponents of surgical robot system 300 including but not limited to abattery 402, a power distribution module 404, a platform interface boardmodule 406, a computer 408, a handle 412, and a tablet drawer 414. Theconnections and relationship between these components is described ingreater detail with respect to FIG. 5 .

FIG. 5 illustrates a block diagram of certain components of an exampleembodiment of surgical robot system 300. Surgical robot system 300 maycomprise platform subsystem 502, computer subsystem 504, motion controlsubsystem 506, and tracking subsystem 532. Platform subsystem 502 mayfurther comprise battery 402, power distribution module 404, platforminterface board module 406, and tablet charging station 534. Computersubsystem 504 may further comprise computer 408, display 304, andspeaker 536. Motion control subsystem 506 may further comprise drivercircuit 508, motors 510, 512, 514, 516, 518, stabilizers 520, 522, 524,526, end-effector 310, and controller 538. Tracking subsystem 532 mayfurther comprise position sensor 540 and camera converter 542. System300 may also comprise a foot pedal 544 and tablet 546.

Input power is supplied to system 300 via a power supply 548 which maybe provided to power distribution module 404. Power distribution module404 receives input power and is configured to generate different powersupply voltages that are provided to other modules, components, andsubsystems of system 300. Power distribution module 404 may beconfigured to provide different voltage supplies to platform interfaceboard module 406, which may be provided to other components such ascomputer 408, display 304, speaker 536, driver circuit 508 to, forexample, power motors 512, 514, 516, 518 and end-effector 310, motor510, ring 324, camera converter 542, and other components for system 300for example, fans for cooling the electrical components within cabinet316.

Power distribution module 404 may also provide power to other componentssuch as tablet charging station 534 that may be located within tabletdrawer 318. Tablet charging station 534 may be in wireless or wiredcommunication with tablet 546 for charging tablet 546. Tablet 546 may beused by a surgeon consistent with the present disclosure and describedherein.

Power distribution module 404 may also be connected to battery 402,which serves as temporary power source in the event that powerdistribution module 404 does not receive power from power supply 548. Atother times, power distribution module 404 may serve to charge battery402 if necessary.

Other components of platform subsystem 502 may also include connectorpanel 320, control panel 322, and ring 324. Connector panel 320 mayserve to connect different devices and components to system 300 and/orassociated components and modules. Connector panel 320 may contain oneor more ports that receive lines or connections from differentcomponents. For example, connector panel 320 may have a ground terminalport that may ground system 300 to other equipment, a port to connectfoot pedal 544 to system 300, a port to connect to tracking subsystem532, which may comprise position sensor 540, camera converter 542, andcameras 326 associated with camera stand 302. Connector panel 320 mayalso include other ports to allow USB, Ethernet, HDMI communications toother components, such as computer 408.

Control panel 322 may provide various buttons or indicators that controloperation of system 300 and/or provide information regarding system 300.For example, control panel 322 may include buttons to power on or offsystem 300, lift or lower vertical column 312, and lift or lowerstabilizers 520-526 that may be designed to engage casters 314 to locksystem 300 from physically moving. Other buttons may stop system 300 inthe event of an emergency, which may remove all motor power and applymechanical brakes to stop all motion from occurring. Control panel 322may also have indicators notifying the user of certain system conditionssuch as a line power indicator or status of charge for battery 402.

Ring 324 may be a visual indicator to notify the user of system 300 ofdifferent modes that system 300 is operating under and certain warningsto the user.

Computer subsystem 504 includes computer 408, display 304, and speaker536. Computer 504 includes an operating system and software to operatesystem 300. Computer 504 may receive and process information from othercomponents (for example, tracking subsystem 532, platform subsystem 502,and/or motion control subsystem 506) in order to display information tothe user. Further, computer subsystem 504 may also include speaker 536to provide audio to the user.

Tracking subsystem 532 may include position sensor 540 and cameraconverter 542. Tracking subsystem 532 may correspond to camera stand 302including camera 326 as described with respect to FIG. 3 . Positionsensor 540 may be camera 326. Tracking subsystem may track the pose ofcertain markers that are located on the different components of system300 and/or instruments used by a user during a surgical procedure. Thistracking may be conducted in a manner consistent with the presentdisclosure including the use of infrared technology that tracks the poseof active or passive elements, such as LEDs or reflective markers,respectively. The pose of structures having these types of markers maybe provided to computer 408 which may be shown to a user on display 304.For example, a surgical instrument 608 having these types of markers andtracked in this manner (which may be referred to as a navigationalspace) may be shown to a user in relation to a three dimensional imageof a patient's anatomical structure.

Motion control subsystem 506 may be configured to physically movevertical column 312, upper arm 306, lower arm 308, or rotateend-effector 310. The physical movement may be conducted through the useof one or more motors 510-518. For example, motor 510 may be configuredto vertically lift or lower vertical column 312. Motor 512 may beconfigured to laterally move upper arm 308 around a point of engagementwith vertical column 312 as shown in FIG. 3 . Motor 514 may beconfigured to laterally move lower arm 308 around a point of engagementwith upper arm 308 as shown in FIG. 3 . Motors 516 and 518 may beconfigured to move end-effector 310 in a manner such that one maycontrol the roll and one may control the tilt, thereby providingmultiple angles that end-effector 310 may be moved. These movements maybe achieved by controller 538 which may control these movements throughload cells disposed on end-effector 310 and activated by a user engagingthese load cells to move system 300 in a desired manner.

Moreover, system 300 may provide for automatic movement of verticalcolumn 312, upper arm 306, and lower arm 308 through a user indicatingon display 304 (which may be a touchscreen input device) the pose of asurgical instrument or component on three dimensional image of thepatient's anatomy on display 304. The user may initiate this automaticmovement by stepping on foot pedal 544 or some other input means.

FIG. 6 illustrates a surgical robot system 600 consistent with anexample embodiment. Surgical robot system 600 may comprise end-effector602, robot arm 604, guide tube 606, instrument 608, and robot base 610.Instrument instrument 608 may be attached to a tracking array 612including one or more tracking markers (such as markers 118) and have anassociated trajectory 614. Trajectory 614 may represent a path ofmovement that instrument 608 is configured to travel once it ispositioned through or secured in guide tube 606, for example, a path ofinsertion of instrument 608 into a patient. In an example operation,robot base 610 may be configured to be in electronic communication withrobot arm 604 and end-effector 602 so that surgical robot system 600 mayassist a user (for example, a surgeon) in operating on the patient 210.Surgical robot system 600 may be consistent with previously describedsurgical robot system 100 and 300.

A tracking array 612 may be mounted on instrument 608 to monitor thepose (e.g., location and orientation) of instrument 608. The trackingarray 612 may be attached to an instrument 608 and may comprise trackingmarkers 804. As best seen in FIG. 8 , tracking markers 804 may be, forexample, light emitting diodes and/or other types of reflective markers(e.g., markers 118 as described elsewhere herein). The tracking devicesmay be one or more line of sight devices associated with the surgicalrobot system. As an example, the tracking devices may be one or morecameras 200, 326 associated with the surgical robot system 100, 300 andmay also track tracking array 612 for a defined domain or relativeorientations of the instrument 608 in relation to the robot arm 604, therobot base 610, end-effector 602, and/or the patient 210. The trackingdevices may be consistent with those structures described in connectionwith camera stand 302 and tracking subsystem 532.

FIGS. 7A, 7B, and 7C illustrate a top view, front view, and side view,respectively, of end-effector 602 consistent with an example embodiment.End-effector 602 may comprise one or more tracking markers 702. Trackingmarkers 702 may be light emitting diodes or other types of active andpassive markers, such as tracking markers 118 that have been previouslydescribed. In an example embodiment, the tracking markers 702 are activeinfrared-emitting markers that are activated by an electrical signal(e.g., infrared light emitting diodes (LEDs)). Thus, tracking markers702 may be activated such that the infrared markers 702 are visible tothe camera 200, 326 or may be deactivated such that the infrared markers702 are not visible to the camera 200, 326. Thus, when the markers 702are active, the end-effector 602 may be controlled by the system 100,300, 600, and when the markers 702 are deactivated, the end-effector 602may be locked in position and unable to be moved by the system 100, 300,600.

Markers 702 may be disposed on or within end-effector 602 in a mannersuch that the markers 702 are visible by one or more cameras 200, 326 orother tracking devices associated with the surgical robot system 100,300, 600. The camera 200, 326 or other tracking devices may trackend-effector 602 as it moves to different positions and viewing anglesby following the movement of tracking markers 702. The pose of markers702 and/or end-effector 602 may be shown on a display 110, 304associated with the surgical robot system 100, 300, 600, for example,display 110 as shown in FIG. 2 and/or display 304 shown in FIG. 3 . Thisdisplay 110, 304 may allow a user to ensure that end-effector 602 is ina desirable position in relation to robot arm 604, robot base 610, thepatient 210, and/or the user.

For example, as shown in FIG. 7A, markers 702 may be placed around thesurface of end-effector 602 so that a tracking device placed away fromthe surgical field 208 and facing toward the robot 102, 301 and thecamera 200, 326 is able to view at least 3 of the markers 702 through arange of common orientations of the end-effector 602 relative to thetracking system 100, 300, 600. For example, distribution of markers 702in this way allows end-effector 602 to be monitored by the trackingdevices when end-effector 602 is translated and rotated in the surgicalfield 208.

In addition, in example embodiments, end-effector 602 may be equippedwith infrared (IR) receivers that can detect when an external camera200, 326 is getting ready to read markers 702. Upon this detection,end-effector 602 may then illuminate markers 702. The detection by theIR receivers that the external camera 200, 326 is ready to read markers702 may signal the need to synchronize a duty cycle of markers 702,which may be light emitting diodes, to an external camera 200, 326. Thismay also allow for lower power consumption by the robotic system as awhole, whereby markers 702 would only be illuminated at the appropriatetime instead of being illuminated continuously. Further, in exampleembodiments, markers 702 may be powered off to prevent interference withother navigation instruments, such as different types of surgicalinstruments 608.

FIG. 8 depicts one type of surgical instrument 608 including a trackingarray 612 and tracking markers 804. Tracking markers 804 may be of anytype described herein including but not limited to light emitting diodesor reflective spheres. Markers 804 are monitored by tracking devicesassociated with the surgical robot system 100, 300, 600 and may be oneor more of the line of sight cameras 200, 326. The cameras 200, 326 maytrack the pose of instrument 608 based on the poses of tracking array612 and markers 804. A user, such as a surgeon 120, may orientinstrument 608 in a manner so that tracking array 612 and markers 804are sufficiently recognized by the tracking device or camera 200, 326 todisplay instrument 608 and markers 804 on, for example, display 110 ofthe example surgical robot system.

The manner in which a surgeon 120 may place instrument 608 into guidetube 606 of the end-effector 602 and adjust the instrument 608 isevident in FIG. 8 . The hollow tube or guide tube 114, 606 of theend-effector 112, 310, 602 is sized and configured to receive at least aportion of the surgical instrument 608. The guide tube 114, 606 isconfigured to be oriented by the robot arm 104 such that insertion andtrajectory for the surgical instrument 608 is able to reach a desiredanatomical target within or upon the body of the patient 210. Thesurgical instrument 608 may include at least a portion of a generallycylindrical instrument. Although a screwdriver is exemplified as thesurgical instrument 608, it will be appreciated that any suitablesurgical instrument 608 may be positioned by the end-effector 602.Further examples of the surgical instrument 608 include one or more of aguide wire, cannula, a retractor, a drill, a reamer, a screwdriver, aninsertion instrument, and a removal instrument. Although the guide tube114, 606 is generally shown as having a cylindrical configuration, theguide tube 114, 606 may have any suitable shape, size and configurationdesired to accommodate the surgical instrument 608 and access thesurgical site.

FIGS. 9A-9C illustrate end-effector 602 and a portion of robot arm 604consistent with an example embodiment. End-effector 602 may furthercomprise body 1202 and clamp 1204. Clamp 1204 may comprise handle 1206,balls 1208, spring 1210, and lip 1212. Robot arm 604 may furthercomprise depressions 1214, mounting plate 1216, lip 1218, and magnets1220.

End-effector 602 may mechanically interface and/or engage with thesurgical robot system and robot arm 604 through one or more couplings.For example, end-effector 602 may engage with robot arm 604 through alocating coupling and/or a reinforcing coupling. Through thesecouplings, end-effector 602 may fasten with robot arm 604 outside aflexible and sterile barrier. In an example embodiment, the locatingcoupling may be a magnetically kinematic mount and the reinforcingcoupling may be a five bar over center clamping linkage.

With respect to the locating coupling, robot arm 604 may comprisemounting plate 1216, which may be non-magnetic material, one or moredepressions 1214, lip 1218, and magnets 1220. Magnet 1220 is mountedbelow each of depressions 1214. Portions of clamp 1204 may comprisemagnetic material and be attracted by one or more magnets 1220. Throughthe magnetic attraction of clamp 1204 and robot arm 604, balls 1208become seated into respective depressions 1214. For example, balls 1208as shown in FIG. 9B would be seated in depressions 1214 as shown in FIG.9A. This seating may be considered a magnetically-assisted kinematiccoupling. Magnets 1220 may be configured to be strong enough to supportthe entire weight of end-effector 602 regardless of the orientation ofend-effector 602. The locating coupling may be any style of kinematicmount that uniquely restrains six degrees of freedom.

With respect to the reinforcing coupling, portions of clamp 1204 may beconfigured to be a fixed ground link and as such clamp 1204 may serve asa five bar linkage. Closing clamp handle 1206 may fasten end-effector602 to robot arm 604 as lip 1212 and lip 1218 engage clamp 1204 in amanner to secure end-effector 602 and robot arm 604. When clamp handle1206 is closed, spring 1210 may be stretched or stressed while clamp1204 is in a locked position. The locked position may be a position thatprovides for linkage past center. Because of a closed position that ispast center, the linkage will not open absent a force applied to clamphandle 1206 to release clamp 1204. Thus, in a locked positionend-effector 602 may be robustly secured to robot arm 604.

Spring 1210 may be a curved beam in tension. Spring 1210 may becomprised of a material that exhibits high stiffness and high yieldstrain such as virgin PEEK (poly-ether-ether-ketone). The linkagebetween end-effector 602 and robot arm 604 may provide for a sterilebarrier between end-effector 602 and robot arm 604 without impedingfastening of the two couplings.

The reinforcing coupling may be a linkage with multiple spring members.The reinforcing coupling may latch with a cam or friction basedmechanism. The reinforcing coupling may also be a sufficiently powerfulelectromagnet that will support fastening end-effector 112 to robot arm604. The reinforcing coupling may be a multi-piece collar completelyseparate from either end-effector 602 and/or robot arm 604 that slipsover an interface between end-effector 602 and robot arm 604 andtightens with a screw mechanism, an over center linkage, or a cammechanism.

Referring to FIGS. 10 and 11 , prior to or during a surgical procedure,certain registration procedures may be conducted in order to trackobjects and a target anatomical structure of the patient 210 both in anavigation space and an image space. In order to conduct suchregistration, a registration system 1400 may be used as illustrated inFIG. 10 .

In order to track the position of the patient 210, a patient trackingdevice 116 may include a patient fixation instrument 1402 to be securedto a rigid anatomical structure of the patient 210 and a dynamicreference array 1404 (also referred to as dynamic reference base (DRB))may be securely attached to the patient fixation instrument 1402. Forexample, patient fixation instrument 1402 may be inserted into opening1406 of dynamic reference array 1404. Dynamic reference array 1404, alsoreferred to as a dynamic reference base, may contain markers 1408 thatare visible to tracking devices, such as tracking subsystem 532. Thesemarkers 1408 may be optical markers or reflective spheres, such astracking markers 118, as previously discussed herein.

Patient fixation instrument 1402 is attached to a rigid anatomy of thepatient 210 and may remain attached throughout the surgical procedure.In an example embodiment, patient fixation instrument 1402 is attachedto a rigid area of the patient 210, for example, a bone that is locatedaway from the targeted anatomical structure subject to the surgicalprocedure. In order to track the targeted anatomical structure, dynamicreference array 1404 is associated with the targeted anatomicalstructure through the use of a registration fixture that is temporarilyplaced on or near the targeted anatomical structure in order to registerthe dynamic reference array 1404 with the pose of the targetedanatomical structure.

A registration fixture 1410 is attached to patient fixation instrument1402 through the use of a pivot arm 1412. Pivot arm 1412 is attached topatient fixation instrument 1402 by inserting patient fixationinstrument 1402 through an opening 1414 of registration fixture 1410.Pivot arm 1412 is attached to registration fixture 1410 by, for example,inserting a knob 1416 through an opening 1418 of pivot arm 1412.

Using pivot arm 1412, registration fixture 1410 may be placed over thetargeted anatomical structure and its pose may be determined in an imagespace and navigation space using tracking markers 1420 and/or fiducials1422 on registration fixture 1410. Registration fixture 1410 may containa collection of markers 1420 that are visible in a navigational space(for example, markers 1420 may be detectable by tracking subsystem 532).Tracking markers 1420 may be optical markers visible in infrared lightas previously described herein. Registration fixture 1410 may alsocontain a collection of fiducials 1422, for example, such as bearingballs, that are visible in an imaging space (for example, a threedimension CT image). As described in greater detail with respect to FIG.11 , using registration fixture 1410, the targeted anatomical structuremay be associated with dynamic reference array 1404 thereby allowingdepictions of objects in the navigational space to be overlaid on imagesof the anatomical structure. Dynamic reference array 1404, located at aposition away from the targeted anatomical structure, may become areference point thereby allowing removal of registration fixture 1410and/or pivot arm 1412 from the surgical area.

FIG. 11 provides example operations 1500 for registration consistentwith the present disclosure. The illustrated operations 1500 begins atstep 1502 wherein an image volume containing a graphical representation(or image(s)) of the targeted anatomical structure may be imported intosystem 100, 300 600, for example computer 408. The image volume may bethree dimensional CT or a fluoroscope scan of the targeted anatomicalstructure of the patient 210 which includes registration fixture 1410and a detectable imaging pattern of markers 1420, e.g., fiducials.

At step 1504, an imaging pattern of markers 1420 (e.g., fiducials) isdetected and registered in the imaging space and stored in computer 408.Optionally, at this time at step 1506, a graphical representation of theregistration fixture 1410 may be overlaid on the images of the targetedanatomical structure.

At step 1508, a navigational pattern of registration fixture 1410 isdetected and registered by recognizing markers 1420. Markers 1420 may beoptical markers that are recognized in the navigation space throughinfrared light by tracking subsystem 532 via position sensor 540. Thus,the pose and other information of the targeted anatomical structure isregistered in the navigation space. Therefore, registration fixture 1410may be recognized in both the image space through the use of fiducials1422 and the navigation space through the use of markers 1420. At step1510, the registration of registration fixture 1410 in the image spaceis transferred to the navigation space. This transferal is done, forexample, by using the relative position of the imaging pattern offiducials 1422 compared to the position of the navigation pattern ofmarkers 1420.

At step 1512, registration of the navigation space of registrationfixture 1410 (having been registered with the image space) is furthertransferred to the navigation space of dynamic registration array 1404attached to patient fixation instrument 1402. Thus, registration fixture1410 may be removed and dynamic reference array 1404 may be used totrack the targeted anatomical structure in both the navigation and imagespace because the navigation space is associated with the image space.

At steps 1514 and 1516, the navigation space may be overlaid on theimage space and objects with markers visible in the navigation space(for example, surgical instruments 608 with optical markers 804). Theobjects may be tracked through graphical representations of the surgicalinstrument 608 on the images of the targeted anatomical structure.

FIGS. 12A-12B illustrate imaging systems 1304 that may be used inconjunction with robot systems 100, 300, 600 to acquire pre-operative,intra-operative, post-operative, and/or real-time image data of patient210. Any appropriate subject matter may be imaged for any appropriateprocedure using the imaging system 1304. The imaging system 1304 may beany imaging device such as imaging device 1306 and/or a C-arm 1308device. It may be desirable to take x-rays of patient 210 from a numberof different positions, without the need for frequent manualrepositioning of patient 210 which may be required in an x-ray system.As illustrated in FIG. 12A, the imaging system 1304 may be in the formof a C-arm 1308 that includes an elongated C-shaped member terminatingin opposing distal ends 1312 of the “C” shape. C-shaped member 1130 mayfurther comprise an x-ray source 1314 and an image receptor 1316. Thespace within C-arm 1308 of the arm may provide room for the physician toattend to the patient substantially free of interference from x-raysupport structure 1318. As illustrated in FIG. 12B, the imaging systemmay include imaging device 1306 having a gantry housing 1324 attached toa support structure imaging device support structure 1328, such as awheeled mobile cart 1330 with wheels 1332, which may enclose an imagecapturing portion, not illustrated. The image capturing portion mayinclude an x-ray source and/or emission portion and an x-ray receivingand/or image receiving portion, which may be disposed about one hundredand eighty degrees from each other and mounted on a rotor (notillustrated) relative to a track of the image capturing portion. Theimage capturing portion may be operable to rotate three hundred andsixty degrees during image acquisition. The image capturing portion mayrotate around a central point and/or axis, allowing image data ofpatient 210 to be acquired from multiple directions or in multipleplanes. Although certain imaging systems 1304 are exemplified herein, itwill be appreciated that any suitable imaging system may be selected byone of ordinary skill in the art.

Turning now to FIGS. 13A-13C, the surgical robot system 100, 300, 600relies on accurate positioning of the end-effector 112, 602, surgicalinstruments 608, and/or the patient 210 (e.g., patient tracking device116) relative to the desired surgical area. In the embodiments shown inFIGS. 13A-13C, the tracking markers 118, 804 are rigidly attached to aportion of the instrument 608 and/or end-effector 112.

FIG. 13A depicts part of the surgical robot system 100 with the robot102 including base 106, robot arm 104, and end-effector 112. The otherelements, not illustrated, such as the display, marker tracking cameras,etc. may also be present as described herein. FIG. 13B depicts aclose-up view of the end-effector 112 with guide tube 114 and aplurality of tracking markers 118 rigidly affixed to the end-effector112. In this embodiment, the plurality of tracking markers 118 areattached to the end-effector 112 configured as a guide tube. FIG. 13Cdepicts an instrument 608 (in this case, a probe 608A) with a pluralityof tracking markers 804 rigidly affixed to the instrument 608. Asdescribed elsewhere herein, the instrument 608 could include anysuitable surgical instrument, such as, but not limited to, guide wire,cannula, a retractor, a drill, a reamer, a screwdriver, an insertioninstrument, a removal instrument, or the like.

When tracking an instrument 608, end-effector 112, or other object to betracked in 3D, an array of tracking markers 118, 804 may be rigidlyattached to a portion of the instrument 608 or end-effector 112.Preferably, the tracking markers 118, 804 are attached such that themarkers 118, 804 are out of the way (e.g., not impeding the surgicaloperation, visibility, etc.). The markers 118, 804 may be affixed to theinstrument 608, end-effector 112, or another object to be tracked, forexample, with an array 612. Usually three or four markers 118, 804 areused with an array 612. The array 612 may include a linear section, across piece, and may be asymmetric such that the markers 118, 804 are atdifferent relative poses with respect to one another. For example, asshown in FIG. 13C, a probe 608A with a 4-marker tracking array 612 isshown, and FIG. 13B depicts the end-effector 112 with a different4-marker tracking array 612.

In FIG. 13C, the tracking array 612 functions as the handle 620 of theprobe 608A. Thus, the four markers 804 are attached to the handle 620 ofthe probe 608A, which is out of the way of the shaft 622 and tip 624.Stereophotogrammetric tracking of these four markers 804 allows theinstrument 608 to be tracked as a rigid body and for the tracking system100, 300, 600 to precisely determine the location of the tip 624 and theorientation of the shaft 622 while the probe 608A is moved around withinview of tracking cameras 200, 326.

To enable automatic tracking of one or more instruments 608,end-effector 112, or other object to be tracked in 3D (e.g., multiplerigid bodies), the markers 118, 804 on each instrument 608, end-effector112, or the like, are arranged asymmetrically with a known inter-markerspacing. The reason for asymmetric alignment is so that it isunambiguous which marker 118, 804 corresponds to a particular pose onthe rigid body and whether markers 118, 804 are being viewed from thefront or back, i.e., mirrored. For example, if the markers 118, 804 werearranged in a square on the instrument 608 or end-effector 112, it wouldbe unclear to the system 100, 300, 600 which marker 118, 804corresponded to which corner of the square. For example, for the probe608A, it would be unclear which marker 804 was closest to the shaft 622.Thus, it would be unknown which way the shaft 622 was extending from thearray 612. Accordingly, each array 612 and thus each instrument 608,end-effector 112, or other object to be tracked should have a uniquemarker pattern to allow it to be distinguished from other instruments608 or other objects being tracked.

Asymmetry and unique marker patterns allow the system 100, 300, 600 todetect individual markers 118, 804 then to check the marker spacingagainst a stored template to determine which instrument 608,end-effector 112, or another object they represent. Detected markers118, 804 can then be sorted automatically and assigned to each trackedobject in the correct order. Without this information, rigid bodycalculations could not then be performed to extract key geometricinformation, for example, such as instrument tip 624 and alignment ofthe shaft 622, unless the user manually specified which detected marker118, 804 corresponded to which position on each rigid body. Theseconcepts are commonly known to those skilled in the operations of 3Doptical tracking.

Turning now to FIGS. 14A-14D, an alternative version of an end-effector912 with moveable tracking markers 918A-918D is shown. In FIG. 14A, anarray with moveable tracking markers 918A-918D are shown in a firstconfiguration, and in FIG. 14B the moveable tracking markers 918A-918Dare shown in a second configuration, which is angled relative to thefirst configuration. FIG. 14C shows the template of the tracking markers918A-918D, for example, as seen by the cameras 200, 326 in the firstconfiguration of FIG. 14A; and FIG. 14D shows the template of trackingmarkers 918A-918D, for example, as seen by the cameras 200, 326 in thesecond configuration of FIG. 14B.

In this embodiment, 4-marker array tracking is contemplated wherein themarkers 918A-918D are not all in fixed position relative to the rigidbody and instead, one or more of the array markers 918A-918D can beadjusted, for example, during testing, to give updated information aboutthe rigid body that is being tracked without disrupting the process forautomatic detection and sorting of the tracked markers 918A-918D.

When tracking any instrument, such as a guide tube 914 connected to theend-effector 912 of a robot system 100, 300, 600, the tracking array'sprimary purpose is to update the pose of the end-effector 912 in thecamera coordinate system. When using the rigid system, for example, asshown in FIG. 13B, the array 612 of reflective markers 118 rigidlyextend from the guide tube 114. Because the tracking markers 118 arerigidly connected, knowledge of the marker poses in the cameracoordinate system also provides exact pose of the centerline, tip, andtail of the guide tube 114 in the camera coordinate system. Typically,information about the pose of the end-effector 112 from such an array612 and information about the pose of a target trajectory from anothertracked source are used to calculate the required moves that must beinput for each axis of the robot 102 that will move the guide tube 114into alignment with the trajectory and move the tip to a particular posealong the trajectory vector. Navigation information can be generatedbased on the calculated moves, which can be displayed for guiding anoperator's movement of the end-effector 112 and/or instrument, and/orcan be provided to one or more motors that can automatically orsemi-automatically cause movement of the end-effector 112.

Sometimes, the desired trajectory is in an awkward or unreachable pose,but if the guide tube 114 could be swiveled, it could be reached. Forexample, a very steep trajectory pointing away from the base 106 of therobot 102 might be reachable if the guide tube 114 could be swiveledupward beyond the limit of the pitch (wrist up-down angle) axis, butmight not be reachable if the guide tube 114 is attached parallel to theplate connecting it to the end of the wrist. To reach such a trajectory,the base 106 of the robot 102 might be moved or a different end-effector112 with a different guide tube attachment might be exchanged with theworking end-effector. Both of these solutions may be time consuming andcumbersome.

As best seen in FIGS. 14A and 14B, if the array 908 is configured suchthat one or more of the markers 918A-918D are not in a fixed positionand instead, one or more of the markers 918A-918D can be adjusted,swiveled, pivoted, or moved, the robot 102 can provide updatedinformation about the object being tracked without disrupting thedetection and tracking process. For example, one of the markers918A-918D may be fixed in position and the other markers 918A-918D maybe moveable; two of the markers 918A-918D may be fixed in position andthe other markers 918A-918D may be moveable; three of the markers918A-918D may be fixed in position and the other marker 918A-918D may bemoveable; or all of the markers 918A-918D may be moveable.

In the embodiment shown in FIGS. 14A and 14B, markers 918A, 918 B arerigidly connected directly to a base 906 of the end-effector 912, andmarkers 918C, 918D are rigidly connected to the guide tube 914. Similarto array 612, array 908 may be provided to attach the markers 918A-918Dto the end-effector 912, instrument 608, or another object to betracked. In this case, however, the array 908 is comprised of aplurality of separate components. For example, markers 918A, 918B may beconnected to the base 906 with a first array 908A, and markers 918C,918D may be connected to the guide tube 914 with a second array 908B.Marker 918A may be affixed to a first end of the first array 908A andmarker 918B may be separated a linear distance and affixed to a secondend of the first array 908A. While first array 908 is substantiallylinear, second array 908B has a bent or V-shaped configuration, withrespective root ends, connected to the guide tube 914, and divergingtherefrom to distal ends in a V-shape with marker 918C at one distal endand marker 918D at the other distal end. Although specificconfigurations are exemplified herein, it will be appreciated that otherasymmetric designs including different numbers and types of arrays 908A,908B and different arrangements, numbers, and types of markers 918A-918Dare contemplated.

The guide tube 914 may be moveable, swivelable, or pivotable relative tothe base 906, for example, across a hinge 920 or another connector tothe base 906. Thus, markers 918C, 918D are moveable such that when theguide tube 914 pivots, swivels, or moves, markers 918C, 918D also pivot,swivel, or move. As best seen in FIG. 14A, guide tube 914 has alongitudinal axis 916 which is aligned in a substantially normal orvertical orientation such that markers 918A-918D have a firstconfiguration. Turning now to FIG. 14B, the guide tube 914 is pivoted,swiveled, or moved such that the longitudinal axis 916 is now angledrelative to the vertical orientation such that markers 918A-918D have asecond configuration, different from the first configuration.

In contrast to the embodiment described for FIGS. 14A-14D, if a swivelexisted between the guide tube 914 and the arm 104 (e.g., the wristattachment) with all four markers 918A-918D remaining attached rigidlyto the guide tube 914 and this swivel was adjusted by the user, therobotic system 100, 300, 600 would not be able to automatically detectthat the guide tube 914 orientation had changed. The robotic system 100,300, 600 would track the positions of the marker array 908 and wouldcalculate incorrect robot axis moves assuming the guide tube 914 wasattached to the wrist (the robot arm 104) in the previous orientation.By keeping one or more markers 918A-918D (e.g., two markers 918C, 918D)rigidly on the guide tube 914 and one or more markers 918A-918D (e.g.,two markers 918A, 918B) across the swivel, automatic detection of thenew position becomes possible and correct robot moves are calculatedbased on the detection of a new instrument or end-effector 112, 912 onthe end of the robot arm 104.

One or more of the markers 918A-918D are configured to be moved,pivoted, swiveled, or the like according to any suitable means. Forexample, the markers 918A-918D may be moved by a hinge 920, such as aclamp, spring, lever, slide, toggle, or the like, or any other suitablemechanism for moving the markers 918A-918D individually or incombination, moving the arrays 908A, 908B individually or incombination, moving any portion of the end-effector 912 relative toanother portion, or moving any portion of the instrument 608 relative toanother portion.

As shown in FIGS. 14A and 14B, the array 908 and guide tube 914 maybecome reconfigurable by simply loosening the clamp or hinge 920, movingpart of the array 908A, 908B relative to the other part 908A, 908B, andretightening the hinge 920 such that the guide tube 914 is oriented in adifferent position. For example, two markers 918C, 918D may be rigidlyinterconnected with the guide tube 914 and two markers 918A, 918B may berigidly interconnected across the hinge 920 to the base 906 of theend-effector 912 that attaches to the robot arm 104. The hinge 920 maybe in the form of a clamp, such as a wing nut or the like, which can beloosened and retightened to allow the user to quickly switch between thefirst configuration (FIG. 14A) and the second configuration (FIG. 14B).

The cameras 200, 326 detect the markers 918A-918D, for example, in oneof the templates identified in FIGS. 14C and 14D. If the array 908 is inthe first configuration (FIG. 14A) and tracking cameras 200, 326 detectthe markers 918A-918D, then the tracked markers match Array Template 1as shown in FIG. 14C. If the array 908 is the second configuration (FIG.14B) and tracking cameras 200, 326 detect the same markers 918A-918D,then the tracked markers match Array Template 2 as shown in FIG. 14D.Array Template 1 and Array Template 2 are recognized by the system 100,300, 600 as two distinct instruments, each with its own uniquely definedspatial relationship between guide tube 914, markers 918A-918D, androbot attachment. The user could therefore adjust the position of theend-effector 912 between the first and second configurations withoutnotifying the system 100, 300, 600 of the change and the system 100,300, 600 would appropriately adjust the movements of the robot 102 tostay on trajectory.

In this embodiment, there are two assembly positions in which the markerarray matches unique templates that allow the system 100, 300, 600 torecognize the assembly as two different instruments or two differentend-effectors. In any position of the swivel between or outside of thesetwo positions (namely, Array Template 1 and Array Template 2 shown inFIGS. 14C and 14D, respectively), the markers 918A-918D would not matchany template and the system 100, 300, 600 would not detect any arraypresent despite individual markers 918A-918D being detected by cameras200, 326, with the result being the same as if the markers 918A-918Dwere temporarily blocked from view of the cameras 200, 326. It will beappreciated that other array templates may exist for otherconfigurations, for example, identifying different instruments 608 orother end-effectors 112, 912, etc.

In the embodiment described, two discrete assembly positions are shownin FIGS. 14A and 14B. It will be appreciated, however, that there couldbe multiple discrete positions on a swivel joint, linear joint,combination of swivel and linear joints, pegboard, or other assemblywhere unique marker templates may be created by adjusting the positionof one or more markers 918A-918D of the array relative to the others,with each discrete position matching a particular template and defininga unique instrument 608 or end-effector 112, 912 with different knownattributes. In addition, although exemplified for end-effector 912, itwill be appreciated that moveable and fixed markers 918A-918D may beused with any suitable instrument 608 or other object to be tracked.

When using an external 3D tracking system 100, 300, 600 to track a fullrigid body array of three or more markers attached to a robot'send-effector 112 (for example, as depicted in FIGS. 13A and 13B), it ispossible to directly track or to calculate the 3D pose of every sectionof the robot 102 in the coordinate system of the tracking cameras 200,326. The geometric orientations of joints relative to the tracker areknown by design, and the linear or angular positions of joints are knownfrom encoders for each motor of the robot 102, fully defining the 3Dpositions of all of the moving parts from the end-effector 112 to thebase 116. Similarly, if a tracker were mounted on the base 106 of therobot 102 (not shown), it is likewise possible to track or calculate the3D position of every section of the robot 102 from base 106 toend-effector 112 based on known joint geometry and joint positions fromeach motor's encoder.

In some situations, it may be desirable to track the poses of allsegments of the robot 102 from fewer than three markers 118 rigidlyattached to the end-effector 112. Specifically, if an instrument 608 isintroduced into the guide tube 114, it may be desirable to track fullrigid body motion of the robot 902 with only one additional marker 118being tracked.

Turning now to FIGS. 15A-15E, an alternative version of an end-effector1012 having only a single tracking marker 1018 is shown. End-effector1012 may be similar to the other end-effectors described herein, and mayinclude a guide tube 1014 extending along a longitudinal axis 1016. Asingle tracking marker 1018, similar to the other tracking markersdescribed herein, may be rigidly affixed to the guide tube 1014. Thissingle marker 1018 can serve the purpose of adding missing degrees offreedom to allow full rigid body tracking and/or can serve the purposeof acting as a surveillance marker to ensure that assumptions aboutrobot and camera positioning are valid.

The single tracking marker 1018 may be attached to the end-effector 1012as a rigid extension to the end-effector 1012 that protrudes in anyconvenient direction and does not obstruct the surgeon's view. Thetracking marker 1018 may be affixed to the guide tube 1014 or any othersuitable pose of on the end-effector 1012. When affixed to the guidetube 1014, the tracking marker 1018 may be positioned at a locationbetween first and second ends of the guide tube 1014. For example, inFIG. 15A, the single tracking marker 1018 is shown as a reflectivesphere mounted on the end of a narrow shaft 1017 that extends forwardfrom the guide tube 1014 and is positioned longitudinally above amid-point of the guide tube 1014 and below the entry of the guide tube1014. This position allows the marker 1018 to be generally visible bycameras 200, 326 but also would not obstruct vision of the surgeon 120or collide with other instruments or objects in the vicinity of surgery.In addition, the guide tube 1014 with the marker 1018 in this positionis designed for the marker array on any instrument 608 introduced intothe guide tube 1014 to be visible at the same time as the single marker1018 on the guide tube 1014 is visible.

As shown in FIG. 15B, when a snugly fitting instrument 608 is placedwithin the guide tube 1014, the instrument 608 becomes mechanicallyconstrained in 4 of 6 degrees of freedom. That is, the instrument 608cannot be rotated in any direction except about the longitudinal axis1016 of the guide tube 1014 and the instrument 608 cannot be translatedin any direction except along the longitudinal axis 1016 of the guidetube 1014. In other words, the instrument 608 can only be translatedalong and rotated about the centerline of the guide tube 1014. If twomore parameters are known, such as (1) an angle of rotation about thelongitudinal axis 1016 of the guide tube 1014; and (2) a position alongthe guide tube 1014, then the position of the end-effector 1012 in thecamera coordinate system becomes fully defined.

Referring now to FIG. 15C, the system 100, 300, 600 should be able toknow when an instrument 608 is actually positioned inside of the guidetube 1014 and is not instead outside of the guide tube 1014 and justsomewhere in view of the tracking cameras 200, 326. The instrument 608has a longitudinal axis or centerline 616 and an array 612 with aplurality of tracked markers 804. The rigid body calculations may beused to determine where the centerline 616 of the instrument 608 islocated in the camera coordinate system based on the tracked position ofthe array 612 on the instrument 608.

The fixed normal (perpendicular) distance D_(F) from the single marker1018 to the centerline or longitudinal axis 1016 of the guide tube 1014is fixed and is known geometrically, and the position of the singlemarker 1018 can be tracked. Therefore, when a detected distance D_(D)from instrument centerline 616 to single marker 1018 matches the knownfixed distance D_(F) from the guide tube axis 1016 (e.g., guide tubecenterline) to the single marker 1018, it can be determined that theinstrument 608 is either within the guide tube 1014 (axis 616, 1016 ofinstrument 608 and guide tube 1014 coincident) or happens to be at somepoint in the locus of possible positions where this distance D_(D)matches the fixed distance D_(F). For example, in FIG. 15C, the normaldetected distance D_(D) from instrument centerline 616 to the singlemarker 1018 matches the fixed distance D_(F) from guide tube axis 1016to the single marker 1018 in both frames of data (tracked markercoordinates) represented by the transparent instrument 608 in twopositions, and thus, additional considerations may be needed todetermine when the instrument 608 is located in the guide tube 1014.

Turning now to FIG. 15D, programmed logic can be used to look for framesof tracking data in which the detected distance D_(D) from instrumentcenterline 616 to single marker 1018 remains fixed at the correct lengthdespite the instrument 608 moving in space by more than some minimumdistance relative to the single sphere 1018 to satisfy the conditionthat the instrument 608 is moving within the guide tube 1014. Forexample, a first frame F1 may be detected with the instrument 608 in afirst position and a second frame F2 may be detected with the instrument608 in a second position (namely, moved linearly with respect to thefirst position). The markers 804 on the instrument array 612 may move bymore than a given amount (e.g., more than 5 mm total) from the firstframe F1 to the second frame F2. Even with this movement, the detecteddistance D_(D) from the instrument centerline vector C′ to the singlemarker 1018 is substantially identical in both the first frame F1 andthe second frame F2.

Logistically, the surgeon 120 or user could place the instrument 608within the guide tube 1014 and slightly rotate it or slide it down intothe guide tube 1014 and the system 100, 300, 600 would be able to detectthat the instrument 608 is within the guide tube 1014 from tracking ofthe five markers (four markers 804 on instrument 608 plus single marker1018 on guide tube 1014). Knowing that the instrument 608 is within theguide tube 1014, all 6 degrees of freedom may be calculated that definethe position and orientation of the end-effector 1012 in space. Withoutthe single marker 1018, even if it is known with certainty that theinstrument 608 is within the guide tube 1014, it is unknown where theguide tube 1014 is located along the instrument's centerline vector C′and how the guide tube 1014 is rotated relative to the centerline vectorC′.

With emphasis on FIG. 15E, the presence of the single marker 1018 beingtracked as well as the four markers 804 on the instrument 608, it ispossible to construct the centerline vector C′ of the guide tube 1014and instrument 608 and the normal vector through the single marker 1018and through the centerline vector C′. This normal vector has anorientation that is in a known orientation relative to the forearm ofthe robot distal to the wrist (in this example, oriented parallel tothat segment) and intersects the centerline vector C′ at a specificfixed position. For convenience, three mutually orthogonal vectors k′,j′, can be constructed, as shown in FIG. 15E, defining rigid bodyposition and orientation of the guide tube 1014. One of the threemutually orthogonal vectors k′ is constructed from the centerline vectorC′, the second vector j is constructed from the normal vector throughthe single marker 1018, and the third vector i′ is the vector crossproduct of the first and second vectors k′, j′. The robot's jointpositions relative to these vectors k′, j′, i′ are known and fixed whenall joints are at zero, and therefore rigid body calculations can beused to determine the pose of any section of the robot relative to thesevectors k′, j′, i′ when the robot is at a home position. During robotmovement, if the positions of the instrument markers 804 (while theinstrument 608 is in the guide tube 1014) and the position of the singlemarker 1018 are detected from the tracking system, and angles/linearpositions of each joint are known from encoders, then position andorientation of any section of the robot can be determined.

In some embodiments, it may be useful to fix the orientation of theinstrument 608 relative to the guide tube 1014. For example, theend-effector guide tube 1014 may be oriented in a particular positionabout its axis 1016 to allow machining or implant positioning. Althoughthe orientation of anything attached to the instrument 608 inserted intothe guide tube 1014 is known from the tracked markers 804 on theinstrument 608, the rotational orientation of the guide tube 1014 itselfin the camera coordinate system is unknown without the additionaltracking marker 1018 (or multiple tracking markers in other embodiments)on the guide tube 1014. This marker 1018 provides essentially a “clockposition” from −180° to +180° based on the orientation of the marker1018 relative to the centerline vector C′. Thus, the single marker 1018can provide additional degrees of freedom to allow full rigid bodytracking and/or can act as a surveillance marker to ensure thatassumptions about the robot and camera positioning are valid.

FIG. 16 is a block diagram of operations 1100 for navigating and movingthe end-effector 1012 (or any other end-effector described herein) ofthe robot 102 to a desired target trajectory. Another use of the singlemarker 1018 on the end-effector 1012 or guide tube 1014 is as part ofthe operations 1100 enabling the automated safe movement of the robot102 without a full tracking array attached to the robot 102. Theseoperations 1100 function when the tracking cameras 200, 326 do not moverelative to the robot 102 (i.e., they are in a fixed position), thetracking system's coordinate system and robot's coordinate system areco-registered, and the robot 102 is calibrated such that the positionand orientation of the guide tube 1014 can be accurately determined inthe robot's Cartesian coordinate system based only on the encodedpositions of each robotic axis.

For these operations 1100, the coordinate systems of the camera basedtracker and the robot should be co-registered, meaning that thecoordinate transformation from the tracking system's Cartesiancoordinate system to the robot's Cartesian coordinate system is needed.For convenience, this coordinate transformation can be a 4×4 matrix oftranslations and rotations that is well known in the field of robotics.This transformation will be termed Tcr to refer to“transformation—camera to robot”. Once this transformation is known, anynew frame of tracking data, which is received as x,y,z coordinates invector form for each tracked marker, can be multiplied by the 4×4 matrixand the resulting x,y,z coordinates will be in the robot's coordinatesystem. To obtain Tcr, a full tracking array on the robot is trackedwhile it is rigidly attached to the robot at a pose that is known in therobot's coordinate system, then known rigid body operations are used tocalculate the transformation of coordinates. It should be evident thatany instrument 608 inserted into the guide tube 1014 of the robot 102can provide the same rigid body information as a rigidly attached arraywhen the additional marker 1018 is also read. That is, the instrument608 need only be inserted to any position within the guide tube 1014 andat any rotation within the guide tube 1014, not to a fixed position andorientation. Thus, it is possible to determine Tcr by inserting anyinstrument 608 with a tracking array 612 into the guide tube 1014 andreading the instrument's array 612 plus the single marker 1018 of theguide tube 1014 while at the same time determining from the encoders oneach axis the current pose of the guide tube 1014 in the robot'scoordinate system.

Logic for navigating and moving the robot 102 to a target trajectory isprovided in the operations 1100 of FIG. 16 . Before entering the loop1102, it is assumed that the transformation Tcr was previously stored.Thus, before entering loop 1102, in step 1104, after the robot base 106is secured, greater than or equal to one frame of tracking data of aninstrument inserted in the guide tube while the robot is static isstored; and in step 1106, the transformation of robot guide tubeposition from camera coordinates to robot coordinates Tcr is calculatedfrom this static data and previous calibration data. Tcr should remainvalid as long as the cameras 200, 326 do not move relative to the robot102. If the cameras 200, 326 move relative to the robot 102, and Tcrneeds to be re-obtained, the system 100, 300, 600 can be made to promptthe user to insert an instrument 608 into the guide tube 1014 and thenautomatically perform the necessary calculations.

In the flowchart of operations 1100, each frame of data collectedincludes the tracked position of the DRB 1404 on the patient 210, thetracked position of the single marker 1018 on the end-effector 1014, anda snapshot of the positions of each robotic axis. From the positions ofthe robot's axes, the pose of the single marker 1018 on the end-effector1012 is calculated. This calculated position is compared to the actualposition of the marker 1018 as recorded from the tracking system. If thevalues agree, it can be assured that the robot 102 is in a known pose.The transformation Tcr is applied to the tracked position of the DRB1404 so that the target for the robot 102 can be provided in terms ofthe robot's coordinate system. The robot 102 can then be commanded tomove to reach the target.

After steps 1104, 1106, loop 1102 includes step 1108 receiving rigidbody information for DRB 1404 from the tracking system; step 1110transforming target tip and trajectory from image coordinates totracking system coordinates; and step 1112 transforming target tip andtrajectory from camera coordinates to robot coordinates (apply Tcr).Loop 1102 further includes step 1114 receiving a single stray markerposition for robot from tracking system; and step 1116 transforming thesingle stray marker from tracking system coordinates to robotcoordinates (apply stored Tcr). Loop 1102 also includes step 1118determining current pose of the single robot marker 1018 in the robotcoordinate system from forward kinematics. The information from steps1116 and 1118 is used to determine step 1120 whether the stray markercoordinates from transformed tracked position agree with the calculatedcoordinates being less than a given tolerance. If yes, proceed to step1122, calculate and apply robot move to target x, y, z and trajectory.If no, proceed to step 1124, halt and require full array insertion intoguide tube 1014 before proceeding; step 1126 after array is inserted,recalculate Tcr; and then proceed to repeat steps 1108, 1114, and 1118.

These operations 1100 have advantages over operations in which thecontinuous monitoring of the single marker 1018 to verify the pose isomitted. Without the single marker 1018, it would still be possible todetermine the position of the end-effector 1012 using Tcr and to sendthe end-effector 1012 to a target pose but it would not be possible toverify that the robot 102 was actually in the expected pose. Forexample, if the cameras 200, 326 had been bumped and Tcr was no longervalid, the robot 102 would move to an erroneous pose. For this reason,the single marker 1018 provides value with regard to safety.

For a given fixed position of the robot 102, it is theoreticallypossible to move the tracking cameras 200, 326 to a new pose in whichthe single tracked marker 1018 remains unmoved since it is a singlepoint, not an array. In such a case, the system 100, 300, 600 would notdetect any error since there would be agreement in the calculated andtracked poses of the single marker 1018. However, once the robot's axescaused the end-effector 102, i.e., guide tube, to move to a new pose,the calculated and tracked poses would disagree and the safety checkwould be effective.

The term “surveillance marker” may be used, for example, in reference toa single marker that is in a fixed pose relative to the DRB 1404. Inthis instance, if the DRB 1404 is bumped or otherwise dislodged, therelative pose of the surveillance marker changes and the surgeon 120 canbe alerted that there may be a problem with navigation. Similarly, inthe embodiments described herein, with a single marker 1018 on therobot's guide tube 1014, the system 100, 300, 600 can continuously checkwhether the cameras 200, 326 have moved relative to the robot 102. Ifregistration of the tracking system's coordinate system to the robot'scoordinate system is lost, such as by cameras 200, 326 being bumped ormalfunctioning or by the robot malfunctioning, the system 100, 300, 600can alert the user and corrections can be made. Thus, this single marker1018 can also be thought of as a surveillance marker for the robot 102.

It should be clear that with a full array permanently mounted on therobot 102 (e.g., the plurality of tracking markers 702 on end-effector602 shown in FIGS. 7A-7C) such functionality of a single marker 1018 asa robot surveillance marker is not needed because it is not requiredthat the cameras 200, 326 be in a fixed position relative to the robot102, and Tcr is updated at each frame based on the tracked position ofthe robot 102. Reasons to use a single marker 1018 instead of a fullarray are that the full array is more bulky and obtrusive, therebyblocking the surgeon's view and access to the surgical field 208 morethan a single marker 1018, and line of sight to a full array is moreeasily blocked than line of sight to a single marker 1018.

Turning now to FIGS. 17A-17B and 18A-18B, instruments 608, such asimplant holders 608B, 608C, are depicted which include both fixed andmoveable tracking markers 804, 806. The implant holders 608B, 608C mayhave a handle 620 and an outer shaft 622 extending from the handle 620.The shaft 622 may be positioned substantially perpendicular to thehandle 620, as shown, or in any other suitable orientation. An innershaft 626 may extend through the outer shaft 622 with a knob 628 at oneend. Implant 10, 12 connects to the shaft 622, at the other end, at tip624 of the implant holder 608B, 608C using typical connection mechanismsknown to those of skill in the art. The knob 628 may be rotated, forexample, to expand or articulate the implant 10, 12. U.S. Pat. Nos.8,709,086 and 8,491,659, which are incorporated by reference herein,describe expandable fusion devices and operations for installation.

When tracking the instrument 608, such as implant holder 608B, 608C, thetracking array 612 may contain a combination of fixed markers 804 andone or more moveable markers 806 which make up the array 612 or isotherwise attached to the implant holder 608B, 608C. The navigationarray 612 may include at least one or more (e.g., at least two) fixedposition markers 804, which are positioned with a known pose relative tothe implant holder instrument 608B, 608C. These fixed markers 804 wouldnot be able to move in any orientation relative to the instrumentgeometry and would be useful in defining where the instrument 608 is inspace. In addition, at least one marker 806 is present which can beattached to the array 612 or the instrument itself which is capable ofmoving within a pre-determined boundary (e.g., sliding, rotating, etc.)relative to the fixed markers 804. The system 100, 300, 600 (e.g., thesoftware) correlates the position of the moveable marker 806 to aparticular position, orientation, or other attribute of the implant 10(such as height of an expandable interbody spacer shown in FIGS. 17A-17Bor angle of an articulating interbody spacer shown in FIGS. 18A-18B).Thus, the system and/or the user can determine the height or angle ofthe implant 10, 12 based on the pose of the moveable marker 806.

In the embodiment shown in FIGS. 17A-17B, four fixed markers 804 areused to define the implant holder 608B and a fifth moveable marker 806is able to slide within a pre-determined path to provide feedback on theimplant height (e.g., a contracted position or an expanded position).FIG. 17A shows the implant 10 (e.g., expandable spacer) at its initialheight, and FIG. 17B shows the implant 10 (e.g., expandable spacer) inthe expanded state with the moveable marker 806 translated to adifferent position. In this case, the moveable marker 806 moves closerto the fixed markers 804 when the implant 10 is expanded, although it iscontemplated that this movement may be reversed or otherwise different.The amount of linear translation of the marker 806 would correspond tothe height of the implant 10. Although only two positions are shown, itwould be possible to have this as a continuous function whereby anygiven expansion height could be correlated to a specific position of themoveable marker 806.

Turning now to FIGS. 18A-18B, four fixed markers 804 are used to definethe implant holder 608C and a fifth, moveable marker 806 is configuredto slide within a pre-determined path to provide feedback on the implantarticulation angle. FIG. 18A shows the articulating spacer 12 at itsinitial linear state, and FIG. 18B shows the spacer 12 in an articulatedstate at some offset angle with the moveable marker 806 translated to adifferent position. The amount of linear translation of the marker 806would correspond to the articulation angle of the implant 12. Althoughonly two positions are shown, it would be possible to have this as acontinuous function whereby any given articulation angle could becorrelated to a specific position of the moveable marker 806.

In these embodiments, the moveable marker 806 slides continuously toprovide feedback about an attribute of the implant 10, 12 based onposition. It is also contemplated that there may be discreet positionsthat the moveable marker 806 must be in which would also be able toprovide further information about an implant attribute. In this case,each discreet configuration of all markers 804, 806 correlates to aspecific geometry of the implant holder 608B, 608C and the implant 10,12 in a specific orientation or at a specific height. In addition, anymotion of the moveable marker 806 could be used for other variableattributes of any other type of navigated implant.

Although depicted and described with respect to linear movement of themoveable marker 806, the moveable marker 806 should not be limited tojust sliding as there may be applications where rotation of the marker806 or other movements could be useful to provide information about theimplant 10, 12. Any relative change in position between the set of fixedmarkers 804 and the moveable marker 806 could be relevant informationfor the implant 10, 12 or other device. In addition, although expandableand articulating implants 10, 12 are exemplified, the instrument 608could work with other medical devices and materials, such as spacers,cages, plates, fasteners, nails, screws, rods, pins, wire structures,sutures, anchor clips, staples, stents, bone grafts, biologics, cements,or the like.

Turning now to FIG. 19A, it is envisioned that the robot end-effector112 is interchangeable with other types of end-effectors 112. Moreover,it is contemplated that each end-effector 112 may be able to perform oneor more functions based on a desired surgical procedure. For example,the end-effector 112 having a guide tube 114 may be used for guiding aninstrument 608 as described herein. In addition, end-effector 112 may bereplaced with a different or alternative end-effector 112 that controlsa surgical device, instrument, or implant, for example.

The alternative end-effector 112 may include one or more devices orinstruments coupled to and controllable by the robot. By way ofnon-limiting example, the end-effector 112, as depicted in FIG. 19A, maycomprise a retractor (for example, one or more retractors disclosed inU.S. Pat. Nos. 8,992,425 and 8,968,363) or one or more mechanisms forinserting or installing surgical devices such as expandableintervertebral fusion devices (such as expandable implants exemplifiedin U.S. Pat. Nos. 8,845,734; 9,510,954; and 9,456,903), stand-aloneintervertebral fusion devices (such as implants exemplified in U.S. Pat.Nos. 9,364,343 and 9,480,579), expandable corpectomy devices (such ascorpectomy implants exemplified in U.S. Pat. Nos. 9,393,128 and9,173,747), articulating spacers (such as implants exemplified in U.S.Pat. No. 9,259,327), facet prostheses (such as devices exemplified inU.S. Pat. No. 9,539,031), laminoplasty devices (such as devicesexemplified in U.S. Pat. No. 9,486,253), spinous process spacers (suchas implants exemplified in U.S. Pat. No. 9,592,082), inflatables,fasteners including polyaxial screws, uniplanar screws, pedicle screws,posted screws, and the like, bone fixation plates, rod constructs andrevision devices (such as devices exemplified in U.S. Pat. No.8,882,803), artificial and natural discs, motion preserving devices andimplants, spinal cord stimulators (such as devices exemplified in U.S.Pat. No. 9,440,076), and other surgical devices. The end-effector 112may include one or instruments directly or indirectly coupled to therobot for providing bone cement, bone grafts, living cells,pharmaceuticals, or another deliverable to a surgical target. Theend-effector 112 may also include one or more instruments designed forperforming a discectomy, kyphoplasty, vertebrostenting, dilation, orother surgical procedure.

The end-effector itself and/or the implant, device, or instrument mayinclude one or more markers 118 such that the pose (e.g., location andposition) of the markers 118 may be identified in three-dimensions. Itis contemplated that the markers 118 may include active or passivemarkers 118, as described herein, that may be directly or indirectlyvisible to the cameras 200. Thus, one or more markers 118 located on animplant 10, for example, may provide for tracking of the implant 10before, during, and after implantation.

As shown in FIG. 19B, the end-effector 112 may include an instrument 608or portion thereof that is coupled to the robot arm 104 (for example,the instrument 608 may be coupled to the robot arm 104 by the couplingmechanism shown in FIGS. 9A-9C) and is controllable by the robot system100. Thus, in the embodiment shown in FIG. 19B, the robot system 100 isable to insert implant 10 into a patient and expand or contract theexpandable implant 10. Accordingly, the robot system 100 may beconfigured to assist a surgeon or to operate partially or completelyindependently thereof. Thus, it is envisioned that the robot system 100may be capable of controlling each alternative end-effector 112 for itsspecified function or surgical procedure.

Although the robot and associated systems described herein are generallydescribed with reference to spine applications, it is also contemplatedthat the robot system is configured for use in other surgicalapplications, including but not limited to, surgeries in trauma or otherorthopedic applications (such as the placement of intramedullary nails,plates, and the like), cranial, neuro, cardiothoracic, vascular,colorectal, oncological, dental, and other surgical operations andprocedures.

Ultrasonic Tracking of Surgical Robot End-Effector and SurgicalInstrument Relative to Patient Image Volume

Numerous embodiments have been described above that utilize opticalbased tracking of markers. Those robotic systems utilized opticaltracking registered to a medical image as feedback for positioning therobotic arm 104 while also displaying graphical representations ofinstruments and anatomical structure captured in patient image volumesto enable user visualization of instrument poses relative to theanatomical structure. Although optical-based tracking can be fast andaccurate, the tracking is interrupted by blockage of line-of-sight fromthe markers, e.g., on patient reference array and/or the robot, to thetracking cameras 200, 326. Additionally, many surgical workflows withthese robotic systems require x-rays or CT scans for operation and/orregistration.

Various embodiments of the present disclosure are directed to using a UStransducer to track the pose of the surgical robot end-effector relativeto patient anatomical structure captured in an image volume. A surgicalrobot system is provided that is positioned relative to anatomicalstructure by US feedback. The surgical robot system may operate withoutoptical tracking or may be configured to operate in conjunction withoptical tracking. As will be explained below, optical tracking may beused to assist in localizing anatomical structure being imaged by a UStransducer and to provide operational redundancy to take over when, forexample, the US transducer ceases to contact the patient and thereforeno longer outputs US imaging data of the anatomical structure.

In one embodiment, a surgical robot system comprises a robot, a UStransducer, and at least one processor. The robot has a robot base, arobot arm coupled to the robot base, and an end-effector coupled to therobot arm, such as explained above in accordance with some embodiments.The end-effector is configured to guide movement of a surgicalinstrument. The US transducer is coupled to the end-effector andoperative to output US imaging data of anatomical structure proximatelylocated to the end-effector. The at least one processor is operative toobtain a 3D image volume, such as MRI or CT, for the patient and totrack pose of the end-effector relative to the anatomical structurecaptured in the image volume based on the US imaging data.

The at least one processor may include one or more data processingcircuits (e.g., microprocessor and/or digital signal processor), whichmay be collocated or distributed across one or more data networks. Theat least one processor is configured to execute program code in one ormore memories to perform some or all of the operations and methods forone or more of the embodiments disclosed herein. The at least oneprocessor may be part of the one or more the controllers disclosedherein.

The end-effector can be located at the distal end of the moving arm andinclude a guide tube through which surgical procedures are performed.

FIG. 20 depicts a guide tube 2000 configured to guide movement of asurgical instrument through the guide tube, and a US transducer unit2010 formed by an array of US transducers spaced apart along a leadingedge of the guide tube 2000. In the example embodiment illustrated inFIG. 20 , the US transducers are spaced apart to form a ring-shape andare at least partially disposed within a leading edge of the guide tube2000. A ring-shaped US transducers layout may be especiallyoperationally accurate because the ring can provide improved USvisualization of anatomical structure, e.g., bone and tissues, that aredistal, medial, and lateral proximately located to the guide tube 2000.

Other configurations of US transducers may be used with the guide tube2000. For example, a plurality of US transducers can spaced apart on theleading edge of the guide tube 2000 or near the leading edge of theguide tube 2000, such as being mounted on a support base that isconnected to the guide tube 2000 or another part of the end-effector.

In one embodiment, the US transducer comprises a planar array of UStransducers that are connected by a mounting arm to the guide tube 2000or another part of the end-effector.

When performing surgery, particularly cranial surgery, the inability totrack the instrument (e.g., probe or tool) tip can leave the surgeonprone to coming into contact with various structures that are not theintended target, therefore risking harm to the patient. By having sometrackable instrument reference able to be located on a live ultrasound,the surgeon has an understanding of the instrument tip location relativeto the point of interest in the image during the procedure.

In accordance with some further embodiments, the US transducer can beconfigured to also sense the position of a surgical instrument that ispassed through the guide tube 2000, such as through the ring-shaped UStransducers 2010.

A US visible reference on a surgical instrument would limit dangersarising if the surgical instrument is not tracked, such as inaccurateinstrument trajectories, instruments appearing to be bending off alongtrajectories, or moving the instrument too deep or shallow relative to adesired location. By utilizing live US while the tracked instrumentprogresses through the surgical site, the instrument's fiducials notonly give information of general positioning relative to the surgicalsite from above, but depending on the type and number of fiducials used,more information can be given. The details of what information isidentifiable in the US imaging data depends on characteristics of thefiducials formed on the tool. One type of fiducial may enable trackingof instrument depth, while a pattern of fiducials may enable tracking ofinstrument rotation and tracking trajectory, such as relating toskiving, bending, etc.

Discrete fiducial features such as protrusions, slots, holes orindentations could be formed on the surface of the shaft of a surgicalinstrument, such as a screwdriver, drill, awl, tap, etc. The UStransducer can be configured to output US imaging data that captureslocations of the discrete features on the surgical instrument andcaptures anatomical structure proximately located to the guide tube2000. At least one processor (also referred to herein as “processor”below for brevity) is operative to identify in the US imaging datalocations of the discrete features which are spaced apart along thesurgical instrument and sensed by the US transducer, and to determinepose of the surgical instrument relative to the end-effector based onthe locations of the discrete features identified in the US imagingdata.

In one embodiment, the processor compares a template of definedlocations of the markings the instrument shaft to the locations ofmarkings identified in the US imaging data, and can determine there fromthe exact longitudinal and rotational position of the surgicalinstrument within the guide tube 2000. The processor may be configuredto graphically display a representation of the surgical instrument witha determined pose overlaid on a graphical representation of anatomicalstructure captured in a medical image volume. This functionality can beadvantageous over systems that require optical or other tracking tovisualize the surgical instrument during insertion.

FIGS. 21A-21C depict differently configured surgical instruments, whichhave shafts configured to be tracked relative to the guide tube 2000using the US transducer.

FIG. 21A depicts a surgical instrument 2100 with patterned indentations2102 at calibrated longitudinal and radial positions on the shaft. Theindentations 2112 are configured to be detectable by the ring-shaped UStransducers 2010 and captured in the US imaging data to enable theprocessor to determine the depth and rotational pose of the surgicalinstrument 2100 within the guide tube 2000 as the instrument shaftpasses through the ring-shaped array of US transducers 2010.

A surgical instrument can have discrete features configured in othermanners to be detectable by the US transducers 2010. In someembodiments, the discrete features are configured as indentations,protrusions, slots, and/or holes spaced apart along a surface of thesurgical instrument.

As explained above, the US transducer can comprise an array of UStransducers. To determine pose of the surgical instrument relative tothe end-effector based on the locations of the discrete featuresidentified in the US imaging data, the processor can be operative todetermine depth of the surgical instrument relative to a location on theend-effector based on counting a number of the discrete featuresidentified in the US imaging data from individual ones of the UStransducers. Alternatively or additionally, when determining pose of thesurgical instrument relative to the end-effector, the processor candetermine rotation of the surgical instrument relative to theend-effector based on identifying rotation of the discrete featuresidentified in the US imaging data between adjacent US transducers in thearray.

In a further embodiment, to determine pose of the surgical instrumentrelative to the end-effector based on the locations of the discretefeatures identified in the US imaging data, the processor is operativeto match a spatial pattern of the locations of the discrete featuresidentified in the US imaging data to content of a template for thesurgical instrument which defines a pattern of the discrete featuresarranged around the surface of the surgical instrument as a function oflocations along a length of the surgical instrument.

In the example of FIG. 21A, the indentations 2102 are formed withalternating patterns 2104 a and 2104 b along a length of the shaft.Pattern 2104 a includes a group of indentations 2102 which arecircumferentially spaced around the shaft and spirally offset in arotational direction along the length of the shaft. In contrast, pattern2104 b includes another group of indentations 2102 which arecircumferentially spaced around the shaft and spirally offset in anopposite rotational direction along the length of the shaft relative tothe pattern 2104 a. The processor can operate to match a spatial patternof the locations of the discrete features identified in the US imagingdata to content of a template for the surgical instrument which definesthe alternating patterns 2104 a and 2104 b, to track the depth androtation of the shaft relative to the guide tube 2000.

FIG. 22 depicts the surgical instrument 2100 of FIG. 21A at threedifferent depths and rotations relative to the guide tube 2000. The poseof the surgical instrument 2100 relative to the guide tube 2000 is beingdetected by the US transducer 2010. Because the indentations 2102 in theinstrument shaft are at calibrated locations along the shaft and have acalibrated spatially shifting pattern that can be matched by theprocessor to a template, the surgical instrument's 2100 depth within theguide tube 2000 can be tracked using the US imaging data. As shown inFIG. 22 , the US signals emitted from each US transducer 2010 fan outand the indentations 2102 in the passing instrument surface reflect backUS signals which are sensed by the US transducers 2010 and captured inthe US imaging data output by the US transducers 2010.

In some other embodiments, the processor is operative to identify in theUS imaging data locations of layers of materials of the surgicalinstrument, where adjacent layers of the materials have differentreflectivity to US. The processor determines pose of the surgicalinstrument relative to the end-effector based on the locations of thelayers of materials of the surgical instrument identified in the USimaging data.

FIG. 21B depicts a surgical instrument 2110 having a shaft withalternating layers of materials, e.g., 2112, 2114, 2112, 2114, andso-on, stacked along a primary axis of the shaft, where adjacent layers2112 and 2114 of the materials have different reflectivity to US. Forexample, layers 2112 may be substantially non-reflective to US andlayers 2114 may be substantially reflective to US. In this manner, thediffering reflectivity of the alternating layers 2112 and 2114 generatesa pattern of US reflections which are identifiable in the US imagingdata from the US transducer. The processor can track depth of thesurgical instrument 2110 relative to the guide tube 2100 based on thepattern. The processor can count the stripes as they go by to determinedepth or spacing between layers could be varied to provide a detectabledepth pattern corresponding to different tool depth within a guide tube.

FIG. 21C depicts a surgical instrument 2110 having a shaft withalternating layers of materials, e.g., 2122, 2124, 2122, 2124, andso-on, forming helical stripes spiraling about a primary axis of theshaft, where adjacent layers 2122 and 2124 of the materials havedifferent reflectivity to US. For example, layers 2122 may besubstantially non-reflective to US and layers 2124 may be substantiallyreflective to US. In this manner, the differing reflectivity of thealternating layers 2122 and 2124 generates a pattern of US reflectionsthat are identifiable in the US imaging data from the US transducer. Theprocessor can track depth and rotation of the surgical instrument 2110relative to the guide tube 2100 based on the pattern. The processor cancount the helix stripes as they pass by the US transducers or the pitch(stripes per cm) of the helix can be configured differently at differentlongitudinal positions to provide markers of specific depths.

In another embodiment, a surgical instrument 2110 has a shaft withlayers of materials forming stripes extending parallel to a primary axisof the shaft, where adjacent layers of the materials have differentreflectivity to US. In this manner, the differing reflectivity of thealternating layers generates a pattern of US reflections which areidentifiable in the US imaging data from the US transducer. Theprocessor can track rotation of the surgical instrument relative to theguide tube 2100 based on the pattern.

Some further embodiments are directed to using US imaging data from a UStransducer in combination with at least one processor (“processor) totrack pose of the robot end-effector relative to anatomical structurecaptured in an image volume for the patient.

FIG. 23 depicts a flowchart of operations that can be performed by aprocessor to track pose of the end-effector relative to anatomicalstructure captured in an image volume based on US imaging data from a UStransducer.

Referring to FIG. 23 , the processor generates 2300 US images of theanatomical structure based on the US imaging data, and matches 2302 theanatomical structure captured in one of the US images to the anatomicalstructure captured in the image volume. The processor then determines2304 the pose of the end-effector relative to the anatomical structurecaptured in the image volume based on the matching and the knownorientation of the end-effector relative to the US transducers.

Some further embodiments are directed to generating steering informationbased on the target pose for surgical instrument in a presently trackedpose of the end-effector relative to the anatomical structure capturedin the image volume, such as according to the flowchart of operationsdepicted in the flowchart of FIG. 24 .

Referring to FIG. 24 , a processor is operative to determine 2400 atarget pose for the surgical instrument based on a surgical plandefining where a surgical procedure is to be performed using thesurgical instrument on the anatomical structure captured in the imagevolume. The processor is further operative to generate 2402 steeringinformation based on the target pose for the surgical instrument and apresent tracked pose of the end-effector relative to the anatomicalstructure captured in the image volume, the steering informationindicating where the surgical instrument and/or the end-effector need tobe moved.

In a further embodiment, the processor is operative to control movementof at least one motor, which is operatively connected to move the robotarm relative to the robot base, based on the steering information toguide movement of the end-effector so the surgical instrument becomespositioned with the target pose

FIG. 25 depicts a more detailed flowchart of operations and be performedby at least one processor (“processor”) to generate navigationinformation that can be used to guide movement of the robot end-effectortoward a target pose, in accordance with some embodiments.

Referring to FIG. 25 , the processor obtains 2400 a 3D MRI or CT imagevolume of the patient. The processor receives 2402 input from a surgeonwho inputs a trajectory plan on anatomical structure captured in animage volume, or the surgical plan may be auto-generated by a surgicalplanning computer and or by the processor. For example, the surgeon mayuse an electronic pen to draw on a graphical representation ofanatomical structure captured in the image volume to input the surgicaltrajectory. The robot arm is manually or automatically positioned 2404by the processor so that the US transducer becomes in contact with thepatient's skin surface. The processor registers 2406 (synchronizescoordinate systems) between coordinate systems of the US transducer, therobot, and the anatomical structure captured in the image volume. Theprocessor displays 2408 a current pose (e.g., position and rotationalorientation) of the end-effector, e.g., guide tube 2000, relative to theanatomical structure captured in the image volume.

The processor determines 2410 whether the end-effector is aligned with atarget pose and, if so, the processor performs further operations 2414associated with being on-target, such as tracking depth and rotation ofa surgical instrument guided by the end-effector. In contrast, when thedetermination 2410 is that the end-effector is not aligned with thetarget pose, the processor generates 2412 navigation informationcomputed to indicate a direction of movement as needed for theend-effector to reach the target pose and initiates further guidedmovement of the end-effector toward the target pose using the navigationinformation.

In one embodiment, the US transducer must remain in contact with thepatient's skin while moving so that the US imaging data from the UStransducer continuously captures anatomical structure of the patientunder the skin. A 6-axis load cell at or near the leading edge of therobot arm may be used to sense pressure of the US transducer and/orend-effector against the patient and ensure that the US transducer staysin gentle contact with the skin. As transitional robot movement occurswhile traveling to the target pose, force feedback at the end-effectorcan be monitored from the load cell and robot arm angle and position canbe responsively adjusted by the processor to maintain a light force onthe skin surface while minimizing shear forces, such as described belowwith regard to FIGS. 26A-C. The force feedback can also be utilized toensure that, during this transitional movement, the US transducerremains normal to the skin surface to provide the clearest imaging ofthe anatomical structures in the US imaging data.

FIGS. 26A-C depicts a sequence of snapshots of a robotic arm 104 of thesurgical robot system 100 moving laterally to a target pose whileautomatically maintaining contact between the US transducer 2010 and thepatient's skin 2600 and normal to the body surface. Responsive to aleading edge of the guide tube 2000 reaching a trajectory at a targetlocation, the processor can operate to automatically adjust the robotarm 104 so that the guide tube 2000 becomes oriented with a pose thatmatches the target trajectory.

Optical tracking may be used to assist in localizing anatomicalstructure being imaged by a US transducer and to provide operationalredundancy to take over when, for example, the US transducer ceases tocontact the patient and therefore no longer outputs US imaging data ofthe anatomical structure.

During a surgical procedure, the surgical robot system 100 may plan orpredict a series of arm movements required to move from a currentposition to a new position with the expectation that the US transducerwill lose contact with the patient's skin and, therefore, ceaseoutputting US imaging data of the anatomical structure which is used fortracking location relative to the anatomical structure captured an imagevolume for the patient. It is further anticipated that the US transducerwill eventually come back in contact with the skin again near a targetlocation and therefore resume outputting US imaging data of theanatomical structure in a region near the target location. As with acontinuous contact mode (e.g., where the US transducer maintains contactpatient's skin), force feedback from one or more sensors can be used tointerrupt controlled movement of the end-effector to ensure safemovement without unexpected collision with the patient or otherobstacle. Processor operations can be configured to enter a “floating”mode in case of detected collision where the robot arm 104 is controlledto be easily moved in any direction with light applied force by a userand/or wait for user intervention.

To clearly indicate to the user when the robot is in contact with thepatient or is unable to determine pose based on US imaging data (e.g.,US transducer has ceased contacting skin) and is estimating where therobot is based on the last known location, the surgical robot system 100can be configured to display anatomical structure in different shades,such as grayscale, and/or different colors to visually differentiatebetween when the US transducer is properly contacting a patient toprovide US imaging data that is being used to identify pose of the UStransducer versus when the US transducer is not satisfying thatcondition. Displaying the anatomical structure in different shadesand/or colors notifies the user when the displayed navigationinformation can be most accurately relied upon for precise navigation(i.e., when relying upon US imaging data of anatomical structurematching anatomical structure captured in the image volume) and when thenavigation information is a rougher estimate (i.e., when not relyingupon such US imaging data) but may still be useful for planning ornon-surgical localization.

In either of these modes (accurate or estimate), once the end-effector112 control by the surgical robot 102 approaches the target location,the surgical robot 102 will adjust the arm 104 orientation to match thedesired trajectory orientation while also monitoring feedback from theload cell. Load feedback would be used to adjust the end-effector 112pose so that the desired orientation is achieved while maintainingconstant low applied force between the US transducer and the patient'sskin.

During any phase of movement where the US transducer is in contact withthe patient's skin, accuracy of the displayed information depends uponrapid re-registration (e.g., matching 2302 in FIG. 23 ) of theanatomical structure captured in the US images generated (2300 in FIG.23 ) based on the US imaging data to the anatomical structure capturedan image volume for the patient. For example, the anatomical structurecaptured in a US image generated based on the US imaging data from theUS detector is rapidly re-registered (matched 2302 in FIG. 23 ) with theanatomical structure captured in the CT or MM scan volume so that thepose of the end-effector 112 (e.g., guide tube 2000) relative to thepatient's anatomical structure is known in near real time. For each ofthe US images, e.g., “frame”, which is generated based on the US imagingdata, the US image is registered to the CT or MRI scan volume, providingan updated computed pose of the US transducer relative to anatomicalstructure captured in the scan (image) volume. Knowing the pose of theend-effector 112 (e.g., guide tube 2000) relative to the US transducerand the pose of the anatomical structure captured in the CT or MRI scanvolume relative to the US transducer, it is then possible to determinethe pose of the end-effector 112 (e.g., guide tube 2000) relative to theanatomical structure captured in the CT or MRI scan volume. Theend-effector 112 (e.g., guide tube 2000) can then be visualized byrendering a graphic representation of the end-effector 112 (e.g., guidetube 2000) relative to (e.g., as a graphical overlay) the anatomicalstructure captured in the CT or MM scan volume. For clear visualizationof the end-effector 112 (e.g., guide tube 2000) relative to theanatomical structure, the scan volume can be displayed as a multiplanarreconstruction (MPR) view showing three mutually orthogonal slice views,two parallel and one perpendicular to the end-effector as is currentlyused in the Globus Excelsius GPS system.

Registration of the pose of the US transducer to the CT volume may becomputationally intensive and have relatively lower reliability if theregistration is not initiated with direction or seeding to a carefullyselected portion of the CT volume, such as if the registrationoperations attempted to look for a match across a large region of the CTvolume. Therefore, the first registration may be computationallyintensive or may require user intervention to achieve desired accuracyor successful completion. However, once the first registration has beencompleted, subsequent re-registrations can be performed with lesscomputational resources needed because the system uses knowledge ofexactly where the end-effector 112 (e.g., guide tube 2000) has moved inits coordinate system via kinematics. When moving to a new targetlocation, the system can assume that the patient anatomy is in a fixedlocation to get within a few millimeters of the target location and thenfocus the registration matching search to within a small range of thepredicted target anatomy for a registration match between the structureof the anatomical structure captured in one of the US images tostructure of the anatomical structure captured in the selected portionof the CT volume. The system can then refine its determination of theend-effector 112 (e.g., guide tube 2000) pose and reach final alignmentbetween target trajectory and end-effector 112 (e.g., guide tube 2000)pose.

In accordance with some further embodiments, the surgical robot systemuses kinematic sensors on the robot, e.g., at pivot joints of the robotarms 104 and end-effector 112, providing kinematic movement data tocontinue to track pose of the end-effector 112 during period while theUS transducer is not outputting US imaging data of the anatomicalstructure, e.g., while the US transducer is lifted not in contact withthe patient. The surgical robot system subsequently resumes using the USimaging data, and may cease any further concurrent use of kinematicmovement data, when the US transducer has again contacted the patientand become re-registered to the CT volume or other image volume for thepatient.

In one embodiment, the surgical robot system includes kinematic sensorsconnected to the robot arm and which are operative to output kinematicmovement data indicating change in pose of the robot arm relative to therobot base. The at least one processor (“processor”) is operative to,after tracking pose of the end-effector relative to the anatomicalstructure captured in the image volume based on the US imaging data fora period of time and responsive to the US transducer ceasing to outputUS imaging data of the anatomical structure proximately located to theend-effector, trigger continued tracking of the pose of the end-effectorrelative to the anatomical structure captured in the image volume basedon the kinematic movement data. The processor is further operative torespond to the US transducer resuming output of US imaging data of theanatomical structure proximately located to the end-effector, bytriggering continued tracking of the pose of the end-effector relativeto the anatomical structure captured in the image volume based on the USimaging data.

In a further related embodiment, the processor may cease tracking of thepose of the end-effector relative to the anatomical structure capturedin the image volume based on the kinematic movement data, responsive tothe US transducer resuming output of US imaging data of the anatomicalstructure proximately located to the end-effector.

In a further related embodiment, the processor can be configured toconstrain the search space for matching the anatomical structurecaptured in one of the US images to the anatomical structure captured inthe image volume, based on a current pose tracked based on the kinematicmovement data (position encoders at each robotic joint). The processorcan operate to trigger continued tracking of the pose of theend-effector relative to the anatomical structure captured in the imagevolume based on the US imaging data, by operations which includegenerating US images of the anatomical structure based on the US imagingdata, selecting a portion of the image volume based on a present pose ofthe end-effector as tracked relative to the anatomical structurecaptured in the image volume based on the kinematic movement data, andmatching structure of the anatomical structure captured in one of the USimages to structure of the anatomical structure captured in the selectedportion of the image volume. The processor determines the pose of theend-effector relative to the anatomical structure captured in theselected portion of the image volume based on the matching.

In some other related embodiments, the surgical robot system uses adifferent color and/or shading to visually indicate to a user when thetracking is performed based on US imaging data distinguished from whenthe tracking is performed based on kinematic movement data. In oneembodiment, the processor is further operative to display a graphicalrepresentation of the end-effector with the determined pose relative toa graphical representation of the anatomy captured in the image volume.The processor uses a different color and/or shading to display thegraphical representation of the end-effector relative to the graphicalrepresentation of the anatomy captured in the image volume to visuallyindicate to a user when the pose of the end-effector relative to theanatomical structure captured in the image volume is being tracked basedon the US imaging data distinguishable by the user from when the pose ofthe end-effector relative to the anatomical structure captured in theimage volume is being tracked based on the kinematic movement data.

Some other embodiments are directed to using machine vision to ensurethat the US transducer remains in contact with the patient's skinsurface while the end-effector is moved to a target pose via adetermined navigated pathway, and while avoiding collisions with otherobjects or body surfaces. The surgical robot system may further utilizemachine learning in combination with machine vision. Visible lightcameras could detect and map the surface of the patient's body and use amachine learning model, such as a neural network model, to determine anoptimal pathway through which the end-effector is to be moved. Forexample, when moving across the spine from left to right, the computeroperations can process the surface map and the starting and targetlocations through a machine learning model that has been trained onspinous (e.g., indicating that skin surface contours rise to a peak andthen descend) and other body geometries to output a preferred navigationpathway for the end-effector to be moved to the target location. Therobot movement would be responsively controlled for the end-effector andUS transducer to rise-up and rotationally angle over the spine and thendecline back down without having to rely solely on force feedback,thereby making the movement smoother and more reliable for maintainingdesired contact between the US transducer and the patient's skin duringthe movement.

Additionally, the prediction of how movement should occur can come fromtransducer feedback and fitting of the patient to a body model. Forexample, the US imaging data from the US transducer may be used toregister the bony anatomy of the patient to an existing CT volume, butthe CT volume may poorly capture the body surface. Accordingly, byfitting the patient's body to a computerized model that is based on age,gender, weight, ethnicity, etc. the body surface contours relative tothe current location of the end-effector can be predicted and used whengenerating the preferred navigation pathway.

In another embodiment, the surgical robot system operates using acombination of optical tracking input and US transducer input. In oneembodiment, the surgical robot system only utilizes the US imaging datafrom the US transducer while the US transducer is close to a targetlocation, e.g., where registration is performed with at least athreshold accuracy. All secondary transitional movement can be guided byoptical feedback.

For example, in some embodiments the surgical robot system switches fromUS tracking to optical tracking responsive to the US transducer ceasingto output US imaging data of the anatomical structure (e.g., losingcontact with the patient).

In one embodiment, the surgical robot system includes a tracking cameraoperative to track pose of markers on the robot arm and/or theend-effector. The at least one processor (“processor”) is operative to,after tracking pose of the end-effector relative to the anatomicalstructure captured in the image volume based on the US imaging data fora period of time and responsive to the US transducer ceasing to outputUS imaging data of the anatomical structure proximately located to theend-effector, trigger continued tracking of the pose of the end-effectorrelative to the anatomical structure captured in the image volume basedon output of the tracking camera. The processor also responds to the UStransducer resuming output of US imaging data of the anatomicalstructure proximately located to the end-effector, by triggeringcontinued tracking of the pose of the end-effector relative to theanatomical structure captured in the image volume based on the USimaging data.

In a further embodiment, the surgical robot system ceases tracking ofthe pose of the end-effector relative to the anatomical structurecaptured in the image volume based on output of the tracking camera.

In another embodiment, the surgical robot system switches from opticaltracking back to US tracking responsive to the US transducer resumingoutput of US imaging data of the anatomical structure (e.g., resumingcontact with the patient).

In one embodiment, the tracking camera operative to capture location ofmarkers on the robot arm and/or the end-effector. The processor isoperative to track pose of the markers. The processor, after trackingpose of the end-effector relative to the anatomical structure capturedin the image volume based on the US imaging data for a period of timeand responsive to the US transducer ceasing to output US imaging data ofthe anatomical structure proximately located to the end-effector, isoperative to trigger continued tracking of the pose of the end-effectorrelative to the anatomical structure captured in the image volume basedon output of the tracking camera. Responsive to the US transducerresuming output of US imaging data of the anatomical structureproximately located to the end-effector, the processor triggerscontinued tracking of the pose of the end-effector relative to theanatomical structure captured in the image volume based on the USimaging data.

In a further embodiment, the surgical robot system ceases tracking ofthe pose of the end-effector relative to the anatomical structurecaptured in the image volume based on output of the tracking camera.

Some further embodiments are directed to the surgical robot systeminitially using optical tracking to track pose of the end-effector whilemoving to a target region of the patient and then switching to trackingpose of the end-effector using US tracking and constraining the searchspace for the matching.

In one embodiment, the surgical robot system includes a tracking cameraoperative to output optical tracking data indicating pose of a referencearray on the robot arm and/or the end-effector and further indicatingpose of a reference array at a defined location on the patient that isapproximately correlated to a defined location in the anatomicalstructure captured in the image volume. The at least one processor(“processor”) is operative to track pose of the end-effector relative tothe anatomical structure captured in the image volume based on theoptical tracking data, while the end-effector is moved toward thepatient for the US transducer to contact the patient. Responsive to theUS transducer contacting the patient and beginning to output US imagingdata of the anatomical structure proximately located to theend-effector, the processor generates US images of the anatomicalstructure based on the US imaging data. The processor selects a portionof the image volume based on a present pose of the end-effector astracked relative to the anatomical structure captured in the imagevolume based on the optical tracking data, and matches structure of theanatomical structure captured in one of the US images to structure ofthe anatomical structure captured in the selected portion of the imagevolume. The processor determines the pose of the end-effector relativeto the anatomical structure captured in the selected portion of theimage volume based on the matching.

In another related embodiment, the processor is operative to determine atarget pose for the surgical instrument based on a surgical plandefining where a surgical procedure is to be performed using thesurgical instrument on the anatomical structure captured in the imagevolume. The processor generates steering information based on the targetpose for the surgical instrument and a present tracked pose of theend-effector relative to the anatomical structure captured in the imagevolume, the steering information indicating where the surgicalinstrument and/or the end-effector need to be moved. The pose of theend-effector relative to the anatomical structure captured in the imagevolume is tracked using the optical tracking data during a time periodwhile the US transducer is not outputting US imaging data of theanatomical structure proximately located to the end-effector. Incontrast, the pose of the end-effector relative to the anatomicalstructure captured in the image volume is tracked using the US imagingdata and without using the optical tracking data during another timeperiod while the US transducer is outputting the US imaging data of theanatomical structure proximately located to the end-effector.

FIG. 27 depicts a flowchart of operations for controlling movement ofthe robot arm to a target pose using a combination of optical feedbackcontrol and US transducer feedback control, in accordance with someembodiments.

Referring to FIG. 27 , the processor obtains 2700 a 3D MRI or CT imagevolume of the patient. The processor receives 2702 input from a surgeonwho inputs a trajectory plan on anatomical structure captured in animage volume, or the surgical plan may be auto-generated by a surgicalplanning computer and or by the processor. For example, the surgeon mayuse an electronic pen to draw on a graphical representation ofanatomical structure captured in the image volume to input the surgicaltrajectory.

The processor performs coarse registration 2704 (synchronizes coordinatesystems) between coordinate systems of the optical tracking system(e.g., tracking cameras 200, 326), the robot, and the anatomicalstructure captured in the image volume. The registration 2704 performedfor optical tracking can be relatively roughly approximate while stillbeing able to obtain successful navigated movement of the end-effectorto a target pose. For example in a difficult case, if the registration2704 has an error of several millimeters or is registered to the wronglevel, the registration error is substantially reduced by furtherre-registration responsive to when the US images are generated from theUS transducer (once the US transducer comes in skin contact) andstructure of the anatomical structure captured in one of the US imagesis matched to structure of the anatomical structure captured in aselected portion of the image volume. Operations can thereforeautomatically adjust optical registration to continuously improveaccuracy once US imaging data capturing anatomical structure of thepatient is obtained from the US transducer.

In the example operational flow, the robot arm is moved 2706 underoptical tracking to be close to the target pose. Responsive to when theUS imaging becomes activated, it is determined that the target pose isactually shifted to the left by 5 mm. Since the end-effector attached tothe robot arm is tracked, the software would then immediately be able toupdate the optical-tracking-to-CT registration to synchronize with thenewly found US-to-CT registration. For example, when the US imagingbecomes activated and registration operations match structure of theanatomical structure captured in one of the US images to structure ofthe anatomical structure captured in the selected portion of the imagevolume, the operations being performing registration 2708 betweencoordinate systems of the US tracking system (e.g., US transducer), therobot, and the anatomical structure captured in the image volume. Theoperations can use the US based registration to improve accuracy of theearlier optical registration between coordinate systems of the opticaltracking system (e.g., tracking cameras 200, 326), the robot, and theanatomical structure captured in the image volume through theregistration operations 2710 using the determined registration betweenthe US tracking system (e.g., US transducer) and the anatomicalstructure captured in the image volume. When initially preformingregistration of the US transducer, the search region in the image volumecan be selected based on the optical tracked location of end-effector.Because US registration is ultimately what is used to perform surgeryand accuracy of the optical system is less important due to its loweraccuracy and being prone to line-of-sight blockage, skin-mounted arraystracked by the tracking cameras to simplify the entire registration andnavigation process. Alternately, visible light tracking of the patient'sbody or of visible markings created on the patient's skin (e.g., usingink) provide adequately accurate optical tracking in this workflow.

The processor displays 2712 a current pose (e.g., position androtational orientation) of the end-effector, e.g., guide tube 2000,relative to the anatomical structure captured in the image volume. Theprocessor determines 2714 whether the end-effector is aligned with atarget pose and, if so, the processor performs further operations 2716associated with being on-target, such as tracking depth and rotation ofa surgical instrument guided by the end-effector. In contrast, when thedetermination 2714 is that the end-effector is not aligned with thetarget pose, the processor generates 2718 navigation informationcomputed to indicate a direction of movement as needed for theend-effector to reach the target pose and initiates further guidedmovement of the end-effector toward the target pose using the navigationinformation.

Hybrid Patient Tracker Utilizing Optical Tracking and US forNoninvasively Tracking Patient Anatomical Structure

As explained above, image-guided surgery often requires an invasivesurgical process exposing bone to mount a patient reference tracker.Exposing the bone can lead to damage to the bone and soft tissues andinfection. It is also time consuming to surgically clear a path to thebone.

Some further embodiments of the present disclosure are directed to ananatomical structure tracker apparatus that includes both a UStransducer and an optical tracking array. In some embodiments, theoptical tracking array includes a plurality of spaced apart markers. TheUS transducer is rigidly coupled to and spaced apart from opticaltracking array, and is operative to output US imaging data of anatomicalstructure. The optical tracking array may be configured as an array of,e.g., 3 or 4 reflective optical markers that are tracked as a rigid bodyby a stereo camera tracking system. Or, the tracking array, which iscombined with the US transducer, can be an electromagnetic sensor thatelectronically streams its 3D position within an electromagnetic field(e.g., Aurora by Northern Digital, Inc.). Additional options exist fortracking such as radiofrequency time-of-flight. The tracking array ismounted rigidly to an array of 1 or more US transducers.

One problem with tracking the spine using camera optical tracking arraysis that the bone must be invasively exposed in order to temporarilyattach an optical tracking array to the bone for monitoring movement ofthe bone, e.g., patient body movement. For example, the optical trackingarray is typically attached to a spinous process clamp or to a spikethat is driven into the ilium or other bony region near the surgicalsite. In contrast using an anatomical structure tracker apparatus inaccordance with various present embodiments, it is possible to mount theoptical tracking array on the skin surface and then to use US transducerrigidly affixed to the optical tracking array to determine pose of theoptical tracking array relative to the bone. When the US tracking isperformed continuously, the movement of the optical tracking array canbe tracked in real time to improve tracking accuracy without requiring arigid interconnection between the bone and the optical tracking array.

FIG. 28 depicts an anatomical structure tracker apparatus that isconfigured in accordance with some embodiments.

Referring to FIG. 28 , the apparatus includes an optical tracking array2810 comprising a plurality of spaced apart markers 2812. Apparatusfurther includes a two-dimensional planar array of US transducers 2802,supported by a rigid base 2800, and connected by a connecting arm to theoptical tracking array 2810. Each of the US transducers 2802 isconfigured to output US pulses and detect returned US reflections of theanatomical structure. Separation between the US transducers 2802 in thearray may preferably be as small as possible to maximize resolution ofthe detected bone surface below the array. A US tracker computer 2820,which may be part of the surgical robot system 100, is configured toreceive US imaging data from the US transducers and generate US imagesof the anatomical structure based on the US imaging data.

The two-dimensional planar array of US transducers 2802 illustrated inFIG. 28 can be, for example, a 7×6 rectangular array of parallel linearUS transducers 2802. Each US transducer 2802 is capable of emitting USpulses and detecting reflected US.

Operations for mounting the US transducers 2802 to skin could useadhesive gel, adhesive tape, elastic bands, or other means. As explainedabove, because of the ability to perform continuous monitoring ofmovement of the optical tracking array 2810 relative to US trackedanatomical structure, e.g., bone, it is not important for the apparatusto be rigidly mounted to bone and is only necessary that it be mountedso that the US transducers 2802 remain in contact with skin and theoptical tracking array 2810 remains in range of and visible to thetracking cameras. Since US transducers generally require gel to conductUS waves from the skin to the probe, a layer of gel could be provided ina center portion of rectangular or ring-shaped adhesive grommets aroundeach individual US transducer to adhere the US transducer to the skinsurface. Alternately, gel could be provided in the center of a largeradhesive rectangular or ring-shaped grommet around the entire array ofUS transducers adhering the array to the skin surface while alsomaintaining a gel pocket between the skin and transducer.

With the apparatus attached to the patient's skin, each US transducercan operate to detect underlying bone and detect the distance to theunderlying bone according to the known speed of sound in the connectivetissue below skin surface and dorsal to the vertebrae. With eachparallel US transducer detecting the closest proximate contour of theunderlying bone, a map of the bony surface could be generated by the UStracker computer 2820.

In some other related embodiments, instead of using an array of parallellinear US transducers, one or more “convex” or “sector” US transducersare used. These US transducers emit US pulses in a fan pattern. Whenutilizing more than one US transducer, some fan planes can be alignedperpendicular to others, such as shown in FIG. 29 . As with the lineararray of US transducers, e.g., as shown in FIG. 28 , such an array wouldalso be able to detect 3D location of bony prominences under the surfaceof the skin based on the known pattern and geometry of the fan planes,but would be able to detect bone over a wider region than the linearprobe array.

FIG. 29 depicts another anatomical structure tracker apparatus that isconfigured in accordance with some other embodiments.

Referring to FIG. 29 , the apparatus includes the optical tracking array2810 comprising the plurality of spaced apart markers 2812. Theapparatus further includes a US transducer having a linear array of UStransducers 2902 each having a major axis and a minor axis, where themajor axes of the US transducers in the linear array are parallel. TheUS transducer also has at least one pair of other US transducers 2904spaced apart on opposite sides of the linear array of US transducers2902. Each of the US transducers 2904 in the at least one pair have amajor axis and a minor axis, where the major axes of the US transducers2904 in the at least one pair are parallel and extend in a directionthat is substantially perpendicular to a direction of the major axes ofthe US transducers 2902 in the linear array. A US tracker computer 2820,which may be part of the surgical robot system 100, is configured toreceive US imaging data from the US transducers 2902 and 2904 andgenerate 3D US images of the anatomical structure based on the USimaging data.

The US transducers 2902 and 2904 are mounted to a base plate 2900. Inthe example of FIG. 29 , linear array of US transducers 2902 has 8convex US transducers which are oriented such that the planes of the fanpattern of emitted US waves from the US transducers 2902 are paralleland are oriented along a first axis of the base plate. There are alsotwo pairs of US transducers 2904 spaced apart on opposite sides of thelinear array of US transducers 2902, and which are oriented such thatthe planes of the fan pattern of emitted US waves from the UStransducers 2904 are oriented along another a second axis of the baseplate which is perpendicular to the first axis. The fan-shaped planes ofUS pulse emission are illustrated in FIG. 29 for two of the eight UStransducers.

As an alternate to either of the US transducers configurationsillustrated in FIGS. 28 and 29 , any 3D US transducer can be used toidentify 3D locations of detected structures relative to the opticaltracking array 2810.

With the surface of a vertebra mapped according using the UStransducer(s), the US tracker computer can track movement of thevertebra. In one embodiment, the bony structures detected by UStransducer can be treated as natural fiducials. That is, a bonyprominence that has a unique structure such as an outcropping or dimplecan be identified automatically and then followed from frame to frame ofUS images generated based on the US imaging data, to keep track of thebone relative to the optical tracking array. If three or more suchnatural fiducials are identified and followed, there is enough data tocompute full rigid body movement of the bone under the skin according toknown operations. In this embodiment, the system does not have anyinformation on what part of the anatomy is being imaged, it is simplyusing the bone as a rigid fixed reference. Therefore, when the patientis in a particular position such as the position at which registrationis recorded, the natural fiducials can be considered to be at their zerolocation. Any movement of the natural fiducials relative to the UStransducer can be tracked essentially by detecting the naturalfiducial's x,y,z location from the linear US transducer. Then at anygiven frame of tracking data containing both optical tracking and UStracking, the vertebra position is the hybrid (optical and US)anatomical structure tracker apparatus position as detected by theoptical data plus the offset as detected based on the US imaging data.

In another embodiment for tracking movement of the vertebra, the USimaging data is used for registration instead of only being used tofollow natural fiducials. That is, the contours of the bony surface asdetected by the US transducer are matched against the known bonycontours from another medical image such as a CT scan. Bony contours inthe CT or MRI image volume are detected using image processing edgedetection algorithms. The medical image becomes registered to thetracker as soon as a unique match between bone contours detected by USand bone contours detected in the medical image volume is determined.That is, when the system identifies a contour match, the transformationto get from the medical image coordinate system to the hybrid (opticaland US) anatomical structure tracker apparatus coordinate system becomesknown. Since the US transducer is in a known position relative to theoptical tracking array, i.e., through the rigid coupling therebetween,the camera coordinate system and the CT coordinate system are thenco-registered. Thus, the hybrid anatomical structure tracker apparatusserves not only as a non-invasive patient reference array, but also as ameans of registration. By using this registration method, no x-rays oradditional ionizing radiation are needed to achieve registration.

The workflow for using the hybrid anatomical structure tracker apparatusin registration and tracking could be as follows, and according to someembodiments. First, the patient receives a 3D scan such as a MRI or a CTscan. Then, in the operating room, a hybrid anatomical structure trackerapparatus is adhered to the skin, superficial to the spine level to beoperated, with a layer of gel captured between the US transducer theskin. The US transducer is activated, detecting the bones underneath andgenerating a surface contour map. An algorithm then compares the surfacecontours as detected from the US transducer to the contours found on thepreoperative MM or CT scan, iteratively comparing different regions atdifferent orientations until a match is found. Once a match is found,registration has been achieved between medical image volume and theoptical/electromagnetic/radiofrequency tracking coordinate system andother tools such as drills, probes, and screwdrivers can be tracked andimages of the tools overlaid on the MRI or CT volume as is commonly donewith surgical navigation. After registration, the hybrid anatomicalstructure tracker apparatus remains in place and serves as a patienttracker, accurately tracking the location of bone by combining opticaltracking data with US tracking data.

In some cases, the hybrid anatomical structure tracker apparatus maybecome obtrusive to the surgeon if it is located directly over the siteat which surgery is being performed. In such cases, the hybridanatomical structure tracker apparatus can be used to register the levelof interest and additional similar trackers could be placed nearby overregions that are less obtrusive but still relatively close to thesurgical site. After registration is established with the primarydevice, the transformation between the secondary tracker(s) and primarytracker can be recorded (“registration transfer” as described elsewhere)and then the primary tracker removed. This method assumes that anymovement of bone at the location where the primary tracker was mountedwould result in equivalent movement of bone at the region where thesecondary tracker is mounted. In cases where large bending of the spinemay occur, a secondary tracker rostral to the primary site and atertiary tracker caudal to the primary site could be used and themovement at the primary site calculated as the average of the secondaryand tertiary movements.

It may be undesirable to apply US continuously for a long period to thepatient. The skin-mounted device could therefore function in differentmodalities. When needed for registration, the device could applycontinuous energy. When monitoring location, the device could pulseintermittently as needed, for example one 100 ms pulse every 2 seconds.Other factors may also be used to trigger when a higher frequency ofsampling is needed. For example, if tracking cameras detect accelerationor movement exceeding some threshold, the system could be put intocontinuous sampling mode until movement ceases. Additionally, if a USpulse detects movement beyond some threshold of the last position, themonitoring algorithm could trigger the system to switch to continuoussampling mode until movement ceases. Finally, an additional sensor suchas an accelerometer sensing movement of the apparatus or pressure sensorsensing a change in pressure of the gel chamber could trigger the systemto enter continuous sampling mode until a stable state is reached again.

Monitoring Sensitive Anatomical Structures During Spine Surgery

In minimally invasive spine surgery (MIS), sequential dilation is usedto gain access from an incision to a surgical target, typically theintervertebral disc space. During dilation the surgeon must monitor thelocation of sensitive structures, such as nerves, veins, and arteries,in order to avoid serious complications caused by compromising thosestructures. The specific structures are dependent on the anatomytraversed in a given approach. The sensitive structures of commonapproaches are described below.

In MIS transforaminal lumbar interbody fusion (TLIF), the intervertebraldisc space is commonly accessed through Kambin's Triangle, an anatomicalcorridor defined by the triangular shape formed by the exiting nerveroot, traversing nerve root, and superior vertebral end plate. Thesurgeon uses this corridor as a safe access space to perform thediscectomy and place the interbody device. Accurate targeting of thiscorridor is crucial to avoid damage to the adjacent nerves.

In lateral lumbar interbody fusion (LLIF), the disc space is commonlyaccessed through a retroperitoneal approach where the dilator is placedposterior to the peritoneum and traversed through the psoas muscle. Inthis approach, the disc space must be accurately targeted withoutviolating the peritoneum and lumbar plexus.

The sensitive anatomical structures are typically monitored throughdirect visualization and/or intraoperative neuromonitoring. Directvisualization involves creating a clear line of sight between thestructure and the surgeon's eyes. This approach typically requires alarger incision and access corridor, which is in opposition withbenefits of minimally invasive surgery.

Intraoperative neuromonitoring is used to identify real-time damage orinsult to nerves by monitoring the electrical activity of the nervoussystem. Stimulated electromyography (EMG) is a common neuromonitoringmodality employed for monitoring the proximity of or irritation toindividual nerve roots associated with motor function during spinesurgery. The system monitors the change in nerve activity relative to anestablished baseline. In some systems, the status is reported as colorindicators which represent grades of change. Less than 100 mA change isreported as a green indicator, greater than 100 mA change is reported asa yellow indicator, and the lack of a response is indicated as a redindicator. This information is limited as it communicates a relative,quantitative status and does not provide intuitive visualization of thenerve location. In addition, neuromonitoring is limited to monitoringnerve activity and is not capable of monitoring blood vessels.

Various further embodiments of the present disclosure are directed todetecting and providing user notification of the location of nerves,blood vessels, and other sensitive structures using intraoperative USimaging. Detecting these structures with US enables the surgeon to beaware of the structure and its location while accessing the disc spacein a minimally invasive approach, rather than relying on largerincisions as in direct visualization or relative status information asin neuromonitoring. Further embodiments are directed to US transducerapparatuses and operations for detecting blood vessels with traditionaland navigated access instruments capable of US imaging.

One US imaging modality for non-invasive visualization of anatomicalstructures, includes nerves and blood vessels. One such application isUS guided nerve block, where US is used to identify the target nerve andguide the needle placement. Another application is the use of Doppler USto measure the amount of blood flow through veins and arteries. DopplerUS may be coupled with US guided nerve block to monitor the position ofa critical vein or artery while the needle is placed. Machine learningcan be implemented to generate 3D models from US scans and to measureand visualize bladder volume.

US transducers are available in a variety beam shapes. Traditionalhandheld US transducers are most commonly convex or linear. Convextransducers contain a curved array of piezoelectric transducers thatemit and receive US signals in a convex beam shape. Similarly, lineartransducers contain a linear array of transducers that emit and receivesignals in a linear beam shape. Endoscopic transducers are significantlysmaller, approximately 2 mm in diameter, for use in endoscopic orendobronchial applications. These probes can be provided with convex andradial beam shapes. In endobronchial US (EBUS) lung biopsy applications,the radial EBUS probe spins to generate a radial beam shape and is usedlocate the tumor in the bronchial tube. The convex EBUS probe is thenused to target the tumor with the biopsy needle.

Some embodiments are directed to US transducer apparatuses andoperations for visualizing sensitive structures in spine surgery bycombining US imaging with traditional and navigated spine accessinstrumentation. These embodiments may replace or supplementneuromonitoring by providing the surgeon the ability to visualize thelocation of nerves relative to instrumentation rather than solelydepending on relative indicators (i.e. red, yellow, green). Theapplication of US Doppler imaging, which can identify fluid movement,can also be used to provide the surgeon the ability to visualize thelocation of veins and arteries relative to instrumentation. In addition,machine learning may be implemented to compute 3D models of spineanatomy, including discectomy volume. Each of the modalities may becombined with navigation to register and augment the US image with CT,MM or fluoroscopic images.

In one embodiment of the present disclosure, a US transducer apparatusincludes a support wire and a US transducer attached to an end of thesupport wire. The support wire may be a rigid support wire, such as a“Kirschner wire” or “K-wire” probe. The US transducer may be one of aconvex US transducer, a radial US transducer, and a linear UStransducer. An interface is provided for communicating US data through aflexible signal wire, which may extend through the support wire, to acomputer configured to process the US data.

The computer may be configured to process the US data to generate agraphical representation of anatomical structure sensed by US signalsemitted by the US transducer. Alternatively or additionally, thecomputer may be configured to process the US data to identify nervesand/or blood vessels within the anatomical structure sensed by USsignals emitted by the US transducer. The computer includes at least oneprocessor and circuitry configured to drive the US transducer togenerate US signal emissions and to condition the return US signalsreceived by the US transducer for processing by the at least oneprocessor.

In spine surgery, long rigid wires, commonly called “Kirschner wires” or“K-wires”, are used to probe anatomy and guide instruments and implantsto the anatomical targets. K-wires are typically guided to the targetusing fluoroscopic imaging. Once the K-wire is placed, larger profileinstruments or implants are guided over the K-wire, which is typicallyanchored in the anatomy. For example, K-wires are used to guidecannulated pedicle screws safely through the pedicle trajectory. K-wiresare also commonly inserted into the intervertebral disc and used toguide sequential dilators and maintain the position of the dilatorrelative to the disc during retractor or port placement.

Various embodiments the present disclosure may remove the need for orsupplement fluoroscopy by allowing the surgeon to monitor the locationof nerves and blood vessels while guiding the K-wire into position. Thepresent wire-like US transducer apparatus and the operationally coupledcomputer are configured to provide visualization of the position ofanatomical structures through 2D US imaging, identifying nerves andblood vessels apart from other anatomy, identifying blood flow using USDoppler imaging, and constructing 3D models of anatomical structuresincluding discectomy volume using 3D US imaging.

As described above, a flexible signal wire is used to transfer the USdata (e.g., US wave signals) to a computer for processing. The supportwire can be flexible to facilitate tip positioning during a procedure.

In some embodiments, a wireless communication interface may be providedbetween the US transducer and the computer, thereby eliminating orreducing the length of the flexible signal wire extending between the UStransducer and a wireless transmitter. In one embodiment, a processor ismounted on the proximal end of the support wire and coupled to the UStransducer to receive US data and further coupled to a wirelesstransmitter to transmit the US data to the computer. As used herein, USdata may be an analog US signal or digital representation thereof. TheUS transducer, the processor, and the wireless transmitter may bepowered by a proximally located battery.

A machine learning model may be used to process the US data to identifyspecific types of anatomical structures. The machine learning model maybe a neural network or other computer algorithm that is trained toidentify the US reflection appearance of specific types of anatomicalstructures. The machine learning model may be trained to differentiateamong learned US reflection appearances of different types of anatomicalstructures, which may be obtained a database, to identify which type ofanatomical structure is likely the source of the observed US reflection.The machine learning model can be trained using data characterizing USreflection appearances of known anatomical structures, which could beprovided through computational modeling of the anatomical structures orthrough expert US users labeling the anatomical structures in US imagesgenerated from US data.

The US transducer apparatus may require rotation during a surgicalprocedure. Rotation at the tip may be manually applied by rotating theentire apparatus or rotating a mechanism at the base of the apparatusthat extends within the support wire, e.g., with rotation of the UStransducer occurring within a sheath across a bearing surface.Alternately, US transducer rotation may be provided using a miniaturemotor mounted near the tip of the support wire, between the support wireand the US transducer, that is powered using, e.g., electrical wiresthat traverse the shaft alongside the US signal wires.

FIG. 30 depicts a US transducer apparatus which includes a support wire3000 connected to a convex US transducer 3002 which communicates USimage data through a flexible signal wire 3004 to a computer, inaccordance with some embodiments. FIG. 30 also depicts another UStransducer apparatus which includes a support wire 3010 connected to aradial US transducer 3012 which communicates US image data through aflexible signal wire 3014 to a computer, in accordance with someembodiments.

FIG. 31 depicts another embodiment of a US transducer apparatus 3100which includes a support wire (which may be similar to a “K-wire”),which is temporarily inserted through a dilator (Cannula) 3104 having aninner void through which the support wire extends and can be removed.The apparatus further includes a convex US transducer 3102 attached to adistal tip of the support wire and/or the cannulated dilator 3104. Aflexible signal wire 3106 or other communication interface carries theUS data from the US transducer 3102 to the computer.

FIG. 31 also depicts another embodiment of a US transducer apparatus3110 which includes a support wire (which may be similar to a “K-wire”),which is temporarily inserted through a dilator (Cannula) 3114 having aninner void through which the support wire extends and can be removed.The apparatus 3110 further includes a radial US transducer 3112 attachedto a distal tip of the support wire and/or the cannulated dilator 3114.A flexible signal wire 3116 or other communication interface carries theUS image data from the US transducer 3112 to the computer.

These removable US transducer apparatuses 3102, 3112 can be connected tothe computer configured to generate the graphical visualizing of theposition of anatomical structures since using ultrasound, to identifynerves and blood vessels apart from other anatomy (which may use machinelearning), to identify blood flow using Doppler imaging, and toconstruct 3D models of anatomical structures including discectomyvolume. The US transducers 3102, 3112 may be attached to the cannulateddilator 3104, 3114 through various mechanisms, including threads,friction, magnets, clamps, etc. The attachment mechanism can include abearing surface or sheath to allow the US transducers 3102, 3112 torotate during imaging as required for radial transducers. A potentialadvantage of one or more of these embodiments is that a US transducerapparatus is provided that can be inserted by a surgeon using a surgicalprocedure developed for rigid wires, such as K-wires.

FIG. 32 illustrates another US transducer apparatus 3200 configured inaccordance with some embodiments. The US transducer apparatus 3200includes a dilator (Cannula) 3214 containing a permanent integratedrigid wire, which may be similar to a “K-wire”. The apparatus 3200further includes a convex US transducer 3212 attached to a distal tip ofthe cannulated dilator 3214. A flexible signal wire 3216 or othercommunication interface carries the US data from the US transducer 3212to the computer. The computer may be configured to process the US datato generate graphical visualization of the position of anatomicalstructures, identify nerves and blood vessels apart from other anatomyusing machine learning, identify blood flow using Doppler imaging, andconstruct 3D models of anatomical structures including discectomyvolume.

FIG. 33 illustrates another US transducer apparatus 3300 configured inaccordance with some embodiments. The US transducer apparatus 3300includes a dilator (Cannula) 3314 containing a permanent integratedrigid wire, which may be similar to a “K-wire”. The apparatus 3300further includes a convex US transducer 3312 is attached to the dilator3314 through a bearing 3314 or sheath which allows rotation of the UStransducer 3312 relative to the dilator 3314. A flexible signal wire3316 or other communication interface carries the US data from the UStransducer 3312 to the computer. The computer may be configured toprocess the US data to generate graphical visualization of the positionof anatomical structures, identify nerves and blood vessels apart fromother anatomy using machine learning, identify blood flow using Dopplerimaging, and construct 3D models of anatomical structures includingdiscectomy volume.

FIG. 34 depicts a US transducer apparatus which includes a support wire3400 and a convex US transducer 3402 connected to the support wire 3400.The convex US transducer 3402 is operable to communicate US data througha flexible signal wire 3406 to a computer. In accordance with someembodiments, an optical tracking array 3404 comprising a plurality ofspaced apart markers is attached to the support wire 3400 at a locationspaced apart from the convex US transducer 3402.

FIG. 34 also depicts another US transducer apparatus which includes asupport wire 3410 and a radial US transducer 3412 connected to thesupport wire 3410. The radial US transducer 3412 is operable tocommunicate US data through a flexible signal wire 3416 to a computer.In accordance with some embodiments, an optical tracking array 3414comprising a plurality of spaced apart markers is attached to thesupport wire 3410 at a location spaced apart from the radial UStransducer 3412.

By integrating an optical tracking array into the US transducerapparatus, the US transducer pose may be tracked using a surgicalnavigation system. Tracking pose of the US transducer enables theprocessor to computationally merge the US image with the primarynavigation image, such as CT, MRI, or fluoroscopy. Navigation alsoallows the US transducer to be graphically represented relative to theprimary navigation image. In addition, by using the optical trackingarray to track the US transducer the computer can be configured togenerate 3D US images. 3D US images can be used as the stand-alonenavigation image or could be used to update the registration of theprimary navigation image.

Updating registration is useful because the US transducer apparatusitself can alter the configuration of anatomical structures throughnormal pressure applied, reducing accuracy of primary rigid bodynavigation. Even if the US transducer apparatus causes changes to thedeep anatomy relative to the surface, by providing a rough locationwithin the image volume of the tool tip, it is computationally easier tofind a match of the US image contours to the CT or MRI image contourswithin the search area defined by navigation than if the entire imagevolume is searched. Finally, integrating navigation is useful becausetracking the US transducer during US scanning facilitates measuringvolumes from the 3D US images. For example, it would be possible tomeasure the size of tumors or the extent of a discectomy. The followingembodiments build on those described above with the addition of opticaltracking arrays, which may be detachable or permanently fixed to the UStransducer apparatus.

FIG. 35 depicts a US transducer apparatus 3500 which includes a supportwire 3503 extending from a flexible signal wire 3508 through a dilator3504 to connect to a convex US transducer 3502. The convex US transducer3502 is operable to communicate US data through the flexible signal wire3508 to a computer. In accordance with some embodiments, an opticaltracking array 3506 comprising a plurality of spaced apart markers isattached to the support wire 3503 at a location spaced apart from theconvex US transducer 3502.

FIG. 35 also depicts another US transducer apparatus 3510 which includesa support wire 3513 extending from a flexible signal wire 3518 through adilator 3514 to connect to a radial US transducer 3512. The radial UStransducer 3512 is operable to communicate US data through the flexiblesignal wire 3518 to a computer. In accordance with some embodiments, anoptical tracking array 3516 comprising a plurality of spaced apartmarkers is attached to the support wire 3513 at a location spaced apartfrom the convex US transducer 3512.

FIG. 36 depicts another US transducer apparatus 3610 which ispermanently integrated into a dilator 3614 connected to a convex UStransducer 3612. The convex US transducer 3612 is operable tocommunicate US data through a flexible signal wire 3618 to a computer.In accordance with some embodiments, an optical tracking array 3616comprising a plurality of spaced apart markers is attached to thedilator 3614 at a location spaced apart from the convex US transducer3612.

FIG. 37 depicts another US transducer apparatus 3700 which ispermanently integrated into a dilator 3714 connected to a radial UStransducer 3712 through a bearing 3713 which allows rotation of theradial US transducer 3712 relative to the dilator 3714. The radial UStransducer 3712 is operable to communicate US data through a flexiblesignal wire 3718 to a computer. In accordance with some embodiments, anoptical tracking array 3716 comprising a plurality of spaced apartmarkers is attached to the dilator 3714 at a location spaced apart fromthe radial US transducer 3712.

Some embodiments of US transducer apparatuses described above haveutilized rigid dilators and support wires. In some other embodiments thesupport wire and dilator are flexible to allow curvature during surgicalprocedures. Allowing curvature can allow more adaptable positioning toreach certain anatomical structures, such as the underside of ribs,while still being trackable using a combination of US and opticaltracking.

FIG. 38 depicts a US transducer apparatus 3800 which includes a dilatoror semi-rigid tube 3814 connected to a radial US transducer 3812 througha bearing 3813 which allows rotation of the radial or convex UStransducer 3812 relative to the dilator 3814. The radial or convex UStransducer 3812 is operable to communicate US data through a flexiblesignal wire 3818 to a computer. In accordance with some embodiments, anoptical tracking array 3816 comprising a plurality of spaced apartmarkers is attached to the dilator 3814 at a location spaced apart fromthe radial US transducer 3812.

Positional tracking of the tip of the US transducer 3812 can beperformed using a sensor that senses the position of the dilator withoutrelying on a rigid extension to extrapolate the tip position from theoptical tracking array 3816. Some embodiments track the tip of the UStransducer 3812 connected to the flexible dilator 3814 usingelectromagnetic tracking, radiofrequency time-of-flight tracking, orfiber-optic tracking to augment optical tracking of the optical trackingarray 3816.

In one embodiment, a fiber optic element extends down a length of theflexible dilator or tube 3814 or other flexible support wire connectedto the US transducer 3812, and is configured to sense variation incurvature of the flexible dilator or tube 3814 or other flexible supportwire. The fiber optic element may include a Fiber Bragg Grating sensor(FBGS) 3820 configured to sense variation in curvature of the flexibledilator or tube 3814 or other flexible support wire. The fiber opticelement can be configured to communicate curvature sensing data throughthe flexible signal wire 3818 to the computer which is configured totrack location of the fiber optic element.

In a further embodiment which uses fiber-optic tracking to augmentoptical tracking, a fiber-optic element such as a FBGS element extendsdown the length of the probe alongside the US signal wires. Usingmethods described for FBGS, the bending of the flexible dilator 3814 totrack the tip position and, thereby, the proximal position of anattached US transducer 3812. After registration of tracking to MM or CTimage volume, the processor can be configured to navigate the UStransducer 3812 within the body, approximating where within the MM or CTimage volume the tip is located. Because the tissue pathway within thebody deforms in response to pressure from the US transducer 3812 andflexible dilator 3814, the position would not be exact. However, theprocessor can use a shape-matching algorithm to determine whichanatomical structures from the MM or CT are being imaged by the UStransducer 3812, further refining the navigated probe tip locationaccuracy.

Further Definitions and Embodiments

In the above-description of various embodiments of present inventiveconcepts, it is to be understood that the terminology used herein is forthe purpose of describing particular embodiments only and is notintended to be limiting of present inventive concepts. Unless otherwisedefined, all terms (including technical and scientific terms) usedherein have the same meaning as commonly understood by one of ordinaryskill in the art to which present inventive concepts belongs. It will befurther understood that terms, such as those defined in commonly useddictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of this specification andthe relevant art and will not be interpreted in an idealized or overlyformal sense expressly so defined herein.

When an element is referred to as being “connected”, “coupled”,“responsive”, or variants thereof to another element, it can be directlyconnected, coupled, or responsive to the other element or interveningelements may be present. In contrast, when an element is referred to asbeing “directly connected”, “directly coupled”, “directly responsive”,or variants thereof to another element, there are no interveningelements present. Like numbers refer to like elements throughout.Furthermore, “coupled”, “connected”, “responsive”, or variants thereofas used herein may include wirelessly coupled, connected, or responsive.As used herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. Well-known functions or constructions may not be described indetail for brevity and/or clarity. The term “and/or” includes any andall combinations of one or more of the associated listed items.

It will be understood that although the terms first, second, third, etc.may be used herein to describe various elements/operations, theseelements/operations should not be limited by these terms. These termsare only used to distinguish one element/operation from anotherelement/operation. Thus, a first element/operation in some embodimentscould be termed a second element/operation in other embodiments withoutdeparting from the teachings of present inventive concepts. The samereference numerals or the same reference designators denote the same orsimilar elements throughout the specification.

As used herein, the terms “comprise”, “comprising”, “comprises”,“include”, “including”, “includes”, “have”, “has”, “having”, or variantsthereof are open-ended, and include one or more stated features,integers, elements, steps, components or functions but does not precludethe presence or addition of one or more other features, integers,elements, steps, components, functions or groups thereof. Furthermore,as used herein, the common abbreviation “e.g.”, which derives from theLatin phrase “exempli gratia,” may be used to introduce or specify ageneral example or examples of a previously mentioned item, and is notintended to be limiting of such item. The common abbreviation “i.e.”,which derives from the Latin phrase “id est,” may be used to specify aparticular item from a more general recitation.

Example embodiments are described herein with reference to blockdiagrams and/or flowchart illustrations of computer-implemented methods,apparatus (systems and/or devices) and/or computer program products. Itis understood that a block of the block diagrams and/or flowchartillustrations, and combinations of blocks in the block diagrams and/orflowchart illustrations, can be implemented by computer programinstructions that are performed by one or more computer circuits. Thesecomputer program instructions may be provided to a processor circuit ofa general purpose computer circuit, special purpose computer circuit,and/or other programmable data processing circuit to produce a machine,such that the instructions, which execute via the processor of thecomputer and/or other programmable data processing apparatus, transformand control transistors, values stored in memory locations, and otherhardware components within such circuitry to implement thefunctions/acts specified in the block diagrams and/or flowchart block orblocks, and thereby create means (functionality) and/or structure forimplementing the functions/acts specified in the block diagrams and/orflowchart block(s).

These computer program instructions may also be stored in a tangiblecomputer-readable medium that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablemedium produce an article of manufacture including instructions whichimplement the functions/acts specified in the block diagrams and/orflowchart block or blocks. Accordingly, embodiments of present inventiveconcepts may be embodied in hardware and/or in software (includingfirmware, resident software, micro-code, etc.) that runs on a processorsuch as a digital signal processor, which may collectively be referredto as “circuitry,” “a module” or variants thereof.

It should also be noted that in some alternate implementations, thefunctions/acts noted in the blocks may occur out of the order noted inthe flowcharts. For example, two blocks shown in succession may in factbe executed substantially concurrently or the blocks may sometimes beexecuted in the reverse order, depending upon the functionality/actsinvolved. Moreover, the functionality of a given block of the flowchartsand/or block diagrams may be separated into multiple blocks and/or thefunctionality of two or more blocks of the flowcharts and/or blockdiagrams may be at least partially integrated. Finally, other blocks maybe added/inserted between the blocks that are illustrated, and/orblocks/operations may be omitted without departing from the scope ofinventive concepts. Moreover, although some of the diagrams includearrows on communication paths to show a primary direction ofcommunication, it is to be understood that communication may occur inthe opposite direction to the depicted arrows.

Many variations and modifications can be made to the embodiments withoutsubstantially departing from the principles of the present inventiveconcepts. All such variations and modifications are intended to beincluded herein within the scope of present inventive concepts.Accordingly, the above disclosed subject matter is to be consideredillustrative, and not restrictive, and the appended examples ofembodiments are intended to cover all such modifications, enhancements,and other embodiments, which fall within the spirit and scope of presentinventive concepts. Thus, to the maximum extent allowed by law, thescope of present inventive concepts are to be determined by the broadestpermissible interpretation of the present disclosure including thefollowing examples of embodiments and their equivalents, and shall notbe restricted or limited by the foregoing detailed description.

What is claimed is:
 1. An anatomical structure tracker apparatuscomprising: an optical tracking array comprising a plurality of spacedapart markers; a US transducer rigidly coupled to and spaced apart fromthe optical tracking array, wherein the US transducer is operative tooutput US imaging data of anatomical structure.
 2. The anatomicalstructure tracker apparatus of claim 1, wherein: the US transducercomprises a two-dimensional planar array of US transducers connected bya mounting arm to the optical tracking array, and each of the UStransducers is configured to output US pulses and detect returned USreflections of the anatomical structure.
 3. The anatomical structuretracker apparatus of claim 1, wherein: the US transducer comprises alinear array of US transducers each having a major axis and a minoraxis, wherein the major axes of the US transducers in the linear arrayare parallel; and at least one pair of other US transducers spaced aparton opposite sides of the linear array of US transducers, each of the UStransducers in the at least one pair having a major axis and a minoraxis, wherein the major axes of the US transducers in the at least onepair are parallel and extend in a direction that is substantiallyperpendicular to a direction of the major axes of the US transducers inthe linear array.
 4. An ultrasound (US) transducer apparatus comprising:a support wire; and a US transducer attached to an end of the supportwire, wherein the support wire comprises a rigid support wire, andfurther comprising: an optical tracking array comprising a plurality ofspaced apart markers, wherein the optically tracking array is attachedto the rigid support wire.
 5. The US transducer apparatus of claim 4,wherein: the support wire comprises a Kirschner wire probe.
 6. The UStransducer apparatus of claim 4, wherein the wire comprises a rigidsupport wire, and further comprising: a rigid cylindrical dilator with alongitudinally extending center void, wherein the rigid support wireextends through the center void of the rigid cylindrical dilator withthe US transducer located outside the center void.
 6. transducerapparatus of claim 6, further comprising: an optical tracking arraycomprising a plurality of spaced apart markers, wherein the opticallytracking array is attached to the rigid cylindrical dilator.
 8. The UStransducer apparatus of claim 6, wherein the US transducer is configuredto be releasably connected to an end of the rigid cylindrical dilator.7. The US transducer apparatus of claim 4, wherein the US transducercomprises one of a convex US transducer, a radial US transducer, and alinear US transducer.
 8. The US transducer apparatus of claim 4, whereinthe support wire comprises a flexible support wire, and furthercomprising: a fiber optic element extending down a length of theflexible support wire and configured to sense variation in curvature ofthe flexible support wire.
 9. The US transducer apparatus of claim 8,wherein: the fiber optic element comprises a Fiber Bragg Gating sensorconfigured to sense variation in curvature of the flexible support wire.10. The US transducer apparatus of claim 8, wherein: the fiber opticelement communicates curvature sensing data through a flexible signalwire to a computer configured to track location of the fiber opticelement.
 11. The US transducer apparatus of claim 4, wherein: the UStransducer communicates US data through a flexible signal wire, whichextends through the support wire, to a computer configured to processthe US data.
 12. The US transducer apparatus of claim 4, wherein: thecomputer is configured to process the US data to generate a graphicalrepresentation of anatomical structure sensed by US signals emitted bythe US transducer.
 13. The US transducer apparatus of claim 4, wherein:the computer is configured to process the US data to identify nervesand/or blood vessels within the anatomical structure sensed by USsignals emitted by the US transducer.
 14. A surgical robot systemcomprising: a robot having a robot base, a robot arm coupled to therobot base, and an end-effector coupled to the robot arm, theend-effector configured to guide movement of a surgical instrument; anultrasound (US) transducer coupled to the end-effector and operative tooutput US imaging data of anatomical structure proximately located tothe end-effector; and an optical tracking array comprising a pluralityof spaced apart markers; a US transducer rigidly coupled to and spacedapart from the optical tracking array, wherein the US transducer isoperative to output US imaging data of anatomical structure at least oneprocessor operative to obtain an image volume for the patient and totrack pose of the end-effector relative to anatomical structure capturedin the image volume based on the US imaging data.
 15. The surgical robotsystem of claim 14, wherein: the US transducer comprises atwo-dimensional planar array of US transducers connected by a mountingarm to the optical tracking array, and each of the US transducers isconfigured to output US pulses and detect returned US reflections of theanatomical structure.
 16. The surgical robot system of claim 14,wherein: the US transducer comprises a linear array of US transducerseach having a major axis and a minor axis, wherein the major axes of theUS transducers in the linear array are parallel; and at least one pairof other US transducers spaced apart on opposite sides of the lineararray of US transducers, each of the US transducers in the at least onepair having a major axis and a minor axis, wherein the major axes of theUS transducers in the at least one pair are parallel and extend in adirection that is substantially perpendicular to a direction of themajor axes of the US transducers in the linear array.